Accuracy of our algorithm¶
To test our algorithms, we have created an automated runner which simulates an environement and is able to perform iterative tests, which means it automatically iterates over a set of given parameters. With that application, you can fast figure out, which parameter-configuration is the best for your environement.
All you need is a backup-file of the Fingerprints which were taken during calibration-phase and a zip of the testpoints(These are also Fingerprints, which would be taken in the same area as the calibration-Fingerprints). There is a GUI availible for this tool. if your interested, check out the file http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/entry/test/ch/hsr/indoorwps/service/LocatorAlgorithmTesterGUI.java in the IndoorWPSPc
The tests were made with the following positioning of the fingerprints:
The following testpoints were used:
The best result was achived with the following configuration:
IN Mac address equals percentage: 0.7 IN Fingerprint count: 7 IN Correlator: PenaltyAddingCorrelator IN Location Calculator: NearestLocationLocationCalculator OUT Floor error: 0 OUT Maximal Error: 29.22790557 OUT Average error: 12.49873015 OUT Minimal Error: 2.151418125
Another test we made had the following Fingerprints as input:
and used the following Fingerprints:
The best results we got here was with this configuration:
IN Mac address equals percentage: 0.1 IN Fingerprint count: 3 IN Correlator: DistanceOrderedFingerprintCorrelator@cdb06e IN Location Calculator: WeightBasedLocationCalculator@1fa1bb6 OUT Floor error: 0 OUT Maximal Error: 11.56884924 OUT Average error: 4.344432564 OUT Minimal Error: 0.836422626
These two tests were all on only one floor, so here is a test that has Fingerprints on two floors. Providing here images would be too much, if you are interested in where the Fingerprints and Testpoints are located, check out the source for those tests http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/show/test/automatedTests/test3
However, these are the results:
IN Mac address equals percentage: 0.1 IN Fingerprint count: 7 IN Correlator: PenaltyAddingCorrelator@1a80a69 IN Location Calculator: WeightBasedLocationCalculator@18c56d OUT Floor error: 0.131357046 OUT Maximal Error: 6.729289119 OUT Average error: 4.680882275 OUT Minimal Error: 1.591537255
As you can see, the results of the last two tests are not that bad. For testseries one, there is definitely room for improovement. However, the Fingerprint we captured for test 1 could be not that accurate as well. There are a lot of possible sources for impreciseness when dealing with WLAN-Localization.
If you want to perform the tests yourtself or take a closer look at the testdata, the files we used are availible at http://dev.ifs.hsr.ch/redmine/projects/indoorwpspc/repository/revisions/master/show/test/automatedTests. The test-data can easily be view in the IndoorWPSPcLocator
One thing we did not take into account in this testseries is that some correlators are parametrizied, like the PenaltyAddingCorrelator. We are working on extending our tool to provide this functionality as well.
We have recently taken the time to capture Fingerprints on our campus. We took approximately 300 Fingerprints, about 2-3 per room. From out experience, it takes 3 to 5 minutes to map a room. Of corse this depends a lot on how big your room is. We have taken our measurements with two devices, an Android G1 as a mobile device and an Acer Aspire 1825PTZ as a notebook. We have uploaded only the data from the G1 to our http://labs.geometa.info/indoorwps/, but you can get the other data as a backup-zip at http://dev.ifs.hsr.ch/indoorwps/browser/IndoorWPSPc/test/automatedTests/hsr_campus/
Go to HSR campusmap to see a map of our campus and get an idea of the test environment.
To create a realistic test environment, we have selected some Fingerprints from the dataset we have captured. We then isolated them and used our optimizationtool to see how accurate the algorithm is.
If you want to perform the tests yourself, you can get them here: http://dev.ifs.hsr.ch/indoorwps/browser/IndoorWPSPc/test/data/test3/
These are the results:
IN Fingerprint count: 5 IN Mac address equals percentage: 0.3 IN Correlator: ch.hsr.indoorwps.domain.correlator.PenaltyAddingCorrelator@1bd747e IN Location Calculator: ch.hsr.indoorwps.domain.locationCalculator.WeightBasedLocationCalculator@17a8913 OUT Floor error: 0.104564393116681 OUT Maximal Error: 24.96396196401327 OUT Average error: 8.203939507025003 OUT Minimal Error: 1.1961130507605513 OUT Error span: 23.767848913252717