Accuracy of our algorithm

To test our algorithms, we have created an automated runner which simulates an environement and is able to perform iterative tests, which means it automatically iterates over a set of given parameters. With that application, you can fast figure out, which parameter-configuration is the best for your environement.

All you need is a backup-file of the Fingerprints which were taken during calibration-phase and a zip of the testpoints(These are also Fingerprints, which would be taken in the same area as the calibration-Fingerprints). There is a GUI availible for this tool. if your interested, check out the file http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/entry/test/ch/hsr/indoorwps/service/LocatorAlgorithmTesterGUI.java in the IndoorWPSPc

Some Tests

The tests were made with the following positioning of the fingerprints:

The following testpoints were used:

The best result was achived with the following configuration:

IN Mac address equals percentage: 0.7
IN Fingerprint count:             7
IN Correlator:                    PenaltyAddingCorrelator
IN Location Calculator:           NearestLocationLocationCalculator
OUT Floor error:                  0
OUT Maximal Error:                29.22790557
OUT Average error:                12.49873015
OUT Minimal Error:                2.151418125

Another test we made had the following Fingerprints as input:

and used the following Fingerprints:

The best results we got here was with this configuration:

IN Mac address equals percentage:    0.1
IN Fingerprint count:                    3
IN Correlator:                            DistanceOrderedFingerprintCorrelator@cdb06e
IN Location Calculator:                    WeightBasedLocationCalculator@1fa1bb6
OUT Floor error:                    0       
OUT Maximal Error:                    11.56884924
OUT Average error:                    4.344432564
OUT Minimal Error:                    0.836422626

These two tests were all on only one floor, so here is a test that has Fingerprints on two floors. Providing here images would be too much, if you are interested in where the Fingerprints and Testpoints are located, check out the source for those tests http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/show/test/automatedTests/test3

However, these are the results:

IN Mac address equals percentage:    0.1
IN Fingerprint count:                    7
IN Correlator:                            PenaltyAddingCorrelator@1a80a69
IN Location Calculator:                    WeightBasedLocationCalculator@18c56d
OUT Floor error:                    0.131357046
OUT Maximal Error:                    6.729289119
OUT Average error:                    4.680882275
OUT Minimal Error:                    1.591537255

As you can see, the results of the last two tests are not that bad. For testseries one, there is definitely room for improovement. However, the Fingerprint we captured for test 1 could be not that accurate as well. There are a lot of possible sources for impreciseness when dealing with WLAN-Localization.

If you want to perform the tests yourtself or take a closer look at the testdata, the files we used are availible at http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/show/test/automatedTests. The test-data can easily be view in the IndoorWPSPcLocator

One thing we did not take into account in this testseries is that some correlators are parametrizied, like the PenaltyAddingCorrelator. We are working on extending our tool to provide this functionality as well.

More testdata

We have recently taken the time to capture Fingerprints on our campus. We took approximately 300 Fingerprints, about 2-3 per room. From out experience, it takes 3 to 5 minutes to map a room. Of corse this depends a lot on how big your room is. We have taken our measurements with two devices, an Android G1 as a mobile device and an Acer Aspire 1825PTZ as a notebook. We have uploaded only the data from the G1 to our http://labs.geometa.info/indoorwps/, but you can get the other data as a backup-zip at http://dev.ifs.hsr.ch/indoorwps/browser/IndoorWPSPc/test/automatedTests/hsr_campus/
Go to HSR campusmap to see a map of our campus and get an idea of the test environment.

To create a realistic test environment, we have selected some Fingerprints from the dataset we have captured. We then isolated them and used our optimizationtool to see how accurate the algorithm is.
If you want to perform the tests yourself, you can get them here: http://dev.ifs.hsr.ch/indoorwps/browser/IndoorWPSPc/test/data/test3/

These are the results:

IN Fingerprint count:    5
IN Mac address equals percentage:    0.3
IN Correlator:    ch.hsr.indoorwps.domain.correlator.PenaltyAddingCorrelator@1bd747e
IN Location Calculator:    ch.hsr.indoorwps.domain.locationCalculator.WeightBasedLocationCalculator@17a8913
OUT Floor error:    0.104564393116681
OUT Maximal Error:    24.96396196401327
OUT Average error:    8.203939507025003
OUT Minimal Error:    1.1961130507605513
OUT Error span:    23.767848913252717

fingerprints_test1.JPG (63.7 KB) Anonymous, 01/19/2011 09:51 AM

fingerprints_test2.jpg (72.7 KB) Anonymous, 01/19/2011 09:51 AM

testpoints_test1.JPG (51.8 KB) Anonymous, 01/19/2011 09:51 AM

testpoints_test2.jpg (67.9 KB) Anonymous, 01/19/2011 09:51 AM