Slashdot Mirror
Testing and Mapping a Cellular Data Network?
bgsneeze writes "In order to resolve an ongoing issue with a vendor, I have been trying to find a way to test different 3G data devices empirically. I would like to be able to chart signal strength, latency, and bandwidth. I would also like to create a map of the coverage area. I have a test 3G card from three different providers. I would like to be able to travel with the setup to several different locations and run tests. What software or techniques would Slashdotters use to test the different devices? Are there any free or open source software packages that will do this?"
hack something in perl or python that is not hard
Do you mean a 3G SIM for use in mobile devices?
If you mean the USB/PC card 3G adapters for computers, I doubt you'll be able to run them on Linux without using WINE or a VM or whatever, which may interfere with latency readings. Those adapters require you to install software(ATT Connection manager in their case) that's only supported on Windows and Mac, and that software is required to "authenticate" your computer and use the network.
Anecdote: I had ATT 3G in the metro San Diego area around 2007. Speeds and coverage were decent enough to browse the web, but too spotty for anything hardcore.
I've also said this a million times, but ATT connection manager takes 10 minutes to install and spins the hard drive like a defrag...waaaay overkill given the reported footprint.
Note: too lazy to dig up hacks and testamonials from Google.
Obviously this can only be used for terrorism.
http://www.cellularmaps.com/coverage_finder.shtml
http://www.mytruecoverage.com/ is a good site to show coverage. You can install an app on your phone then manually run tests. The results usually take 12-36 hours to post to the website.
I assume that you are not working with a network operator? They have plenty of tools to do drives and plot out signal. Also, chip makers have these tools. I used to work for a large mobile phone chip maker and we had internal test setups that can be driven around in a car to plot signal, etc. Also, we developed a small device FPGA based device that could be tuned to various channels and recover the "sync" and "paging" channels of a CDMA system. You could do the same for UMTS, EVDO, etc. If you just wanted raw signal strength, all you really need is dBm for whatever provider and tower you are looking at. You could then compare strength between providers. However, this isn't really apples to apples. CDMA based systems can literally pick up signal out of horrid conditions and low signal powers... much than, say, a GSM based system. So if you were comparing purely 3G to 3G, that would be valid, but not in the 2G realm. Just get a phone for each provider, turn on the engineering mode to get the phone's reported dBm reading, and then plot the values as you drive around....
iperf for doing throughput tests.
Empirical testing with wireless CELLULAR networks can be very tricky; A few things that you will need to keep in mind is that for testing, you want to make sure your transceiver setup is perfectly reproducable; Same card, same ANTENNA, same position of antenna. When performing your testing, your signal strength will depend on several factors: Distance from the site, antenna type/gain, and specifically what sector/node on the site you happen to be on. while driving in a straight line, you may find that you approach steep nulls near the border of a cells sector boundary. Alot depends on the ability of your particular card to handle the handoff between sectors/sites.
Your latency measurements will also vary according to the individual usage of the sector/site that you are currently on, and additionally vary with time and also variable bandwidth allocation to the sites from the main switch.
There are quite a few test sets and software suited that are commercially available and are tailored for this use, and are used heavily by the mobile data/cellular industry in their drive testing and coverage verification methodologies.
Good luck to you on your testing.
--ToO
Don't know any software built for this, but the key tools would be:
Latency:
* ping / traceroute (if you don't know what these are...)
Bandwidth:
* wget/curl/whatever (fetch a file, time it, get end-to-end bandwidth)
* bing -- less-known, checks bandwidth by measuring the difference in RTT for different-sized payloads. Can measure bandwidth of a particular segment of the route, so e.g. terminal-to-tower vs. backhaul measurements.
Signal strength (and other connection parameters!) -- IIRC there's AT codes for this, but I'm really fuzzy on this end.
Logging/visualization:
* You'll probably want the data crammed into a GPX at some point, but I'd just log it in a flat file. This is really as simple as a shell-script loop running the various tests, fetching the gps position, and writing it out. No need for any synchronization, or loop delays -- just run the tests, format and write one line of results, and repeat.
* Process the results to pretty graphs later. I'd probably hack something together in GNU Octave/MATLAB, and it would be way to slow, and then I'd rewrite it in C because that's easier than vectorizing it properly in MATLAB, and _then_ I'd google around and find out that somebody else had already implemented a general purpose GIS data visualization system well suited to the project -- so don't be like me, try googling first. ;)
What software or techniques would Slashdotters use to test the different devices? Are there any free or open source software packages that will do this?
I would suggest the "grad student technique".
Find a nearby university and convince a professor to lend some (under)grad students to your cause.
It helps if you have friends that can refer you to professors/students who'd be interested in the challenge.
P.S. The only thing better than the "grad student technique" is the "summer intern technique"
[Fuck Beta]
o0t!
Many cellphones have test/debug modes that will give you all kinds of data regarding signal quality and strength. Can't say for bandwidth / latency though. You could check some of the phone hacking forums for a debug mode / debug drivers for your card(s), and see what kind of data you could get to.
There's plenty of network monitoring tools around.
Signal strenght might be harder if the manufacturer of the device doesn't give easy access to that data (many cellphones can be switched into diagnostic mode; or some smartphone OSes have diagnostic software available). Maybe you can find some electrosensitive person?
One that hath name thou can not otter
There is the Placelab.org project, but their mailing list has died down over the years.
Zhrodague.net - I do projects and stuff too.
I am not aware of any tools that would work out of box but Kismet for instance has been used to do this on wifi points as well as a slew of other tools that can map wifi hotspots. The only real difference is the frequencies and communications protocols and you should be able to get going in no time. Google, "Kismet Mapping" for some examples.
I doubt any tests you come up with will be relevant in any way at all. Given variances in reception from device instance to device instance let alone (presumably) three different devices (from three manufacturers on two different technologies?) the reception will be variable. One could ask the store clerk which modem has the best reception to even the odds, but one could also ask him to come up with a means of developing cheap and efficient nuclear fusion - he's a ^%*%@ing store clerk which means he's a high school kid or an adult who can't get a better job, not the CTO of a multi-billion dollar company.
Then we get throughput testing. Where's the bottleneck? Is it the card, your PC, the reception, the time of day, the number of users on the network, the mobile network, the mobile backhaul, the carrier's IP network, the Internet itself, or the server you're downloading data from. Just by luck of peering connecting to Speedtest.net on Verizon at a particular location could be 12 hops away and from AT&T it could be 3. Move two blocks down and the numbers could be reversed.
Now, if you can convince everyone on every carrier to send tracking data including GPS location, carrier position, presence of 3G network, signal strength, latency, and bandwidth tests then you're getting somewhere. But one guy with a laptop and a dream of taking down a carrier isn't going to fly unless you use the Chewbacca Defense.
Can you ping me yet?
How about now?
At least, this is what I see working with SNMP data coming off dumps from the cellular base transceiver stations attached to our towers.
Every 15 minutes, every GSM handset measures the perceived strength of the tower signal it is using, this is reported to the tower, and we record them all. Also every 15 minutes, the strength of the handset signal, from the perspective of the tower is sampled and recorded.
These readings go into buckets, for example if a reading showed a -78 dbm signal, it goes in the -78dbm bucket. Before long, a histogram can be generated with the datain the buckets, and we can see typical distribution for receive performance - for both handsets and the tower. Different towers have very different "signatures" in this data.
You may go around sampling receive performance, (which would be interesting) but I don't think you'll be able to map how well the cellular system is receiving from you.
Most providers use modelled data to estimate signal strength as it's too expensive to actually measure every point within their coverage. You might be able to obtain and use modelled coverage (which is mostly based on putting virtual towers into a digital terrain model) and undertake some validation. This would provide a meaure of the optimism usually present in modelled coverage. Actual coverage is usually going to be less due to buildings, weather, trees and other rubbish you rarely see in a DTM. Validation of published coverage maps would also require far fewer observations ("field work") than creating your own map from scratch.
Suck the whole lot into Quantum GIS (FOSS) and generate a text dump of obesrved/modelled coverage to analyse statistically in R (also FOSS).
Xix.
"Everything is adjustable, provided you have the right tools"
After what happened to wardriving be sure you legally have your butt covered. Especially if you prove the network sucks >.>
I would suggest learning that it's pointless to test for something you can't fix. You'll waste much less time and energy.
Your provider won't care even a Tiny Iota about your results. If you show that their network is fine, they'll ignore you. Even if you prove their network is not fine and all the packets are routed though a Sinclair ZX80 in Clive's basement they'll still ignore you. If you're really lucky, you might get a nice, polite "Fuck you very much, We're not fixing it."
Knowledge is knowing that they have 3000ms latency between nodes. Wisdom is knowing that the only thing you can do is vote with your wallet.
I wouldn't worry about charting the signal strength for 3G. You can be in a densely populated area showing five bars of 3G and your speed and latency can still be dog shit depending on how many people are hitting the tower, similar to your cable modem. It might be worth it to record whether or not you have 3G just to help map out your general coverage, but that doesn't mean you'll have great speed. Although, you can find something like that here.
As for speed I like to use a util called iperf for measuring speed from one device to another across a network. You may have to open ports on your firewall or setup a VPN, which will add unwanted overhead, but you will get a good idea of which carriers have the best speeds. You can also run the simple tests using other websites like here or here.
Loading...
I've had to do this sort of testing at my previous employer to get a new installation accepted by the customer.
One of the biggest problems you will run into is network congestion, can you say with 100% certainty where slow downs are occurring. When we did our testing we limited each base station to a single mobile station so that the radio link would not be congested and then we had a huge pipe to a local server running various traffic generators for us, on occasion we plugged the test server directly into the switch which the base station was connected.
All of this became very difficult once they started adding customers with a 95 to 1 over subscription ratio
Also with other things you measure and record you will want to record signal to noise, signal to interference, co-channel and adjacent channel signal levels, etc....
Industry standard?
http://www.qacafe.com/cdrouter
I've done similar things using Linux. Basically, when you hook up the 3G card, it presents itself as an old-school modem (although it's over USB). You can then use a program such as wvDial or Minicom to send AT commands (if you want to do a connect, you need to find out the number to dial. You can find this on your carrier's website).
A lot of wireless modems have AT commands for signal strength and other info, you can do a Google search for documentation about AT commands for your specific modem. A warning: writing a robust, industrial quality parser for AT commands is one of the most difficult things I've ever programmed. A lot of people take the easy way out, and if a command doesn't work, they just send it a couple more times until it does (which is good enough for many tasks).
If that doesn't provide enough info for you, and you have money, then you should contact the manufacturer. Some modems, like Qualcomm modems, have a separate diagnostic port in a proprietary protocol that provides way more information than you ever dreamed existed, but you will have to pay to find out the protocol.
Qxe4
Working for a telecom company I have used industry tools that do just this. Unfortunately the tools we use are very expensive and proprietary due to Qualcomm's hold on the EvDO IP. But for EvDO based devices we use Qualcomm's QXDM to diagnose the mobile RF along with a GPS receiver. Then we usually use Actix's Analyzer to post-process the data onto a map.
If a telco asks you to prove that signal strength is a problem that's because they don't expect you to get an answer. They're gaming you; fobbing you off; putting the ball in your court. Even if you got a reasonable answer at your own time and expense then nothing will happen. Now is a very good time to switch telco and change all of your bank account numbers.
If you're on Linux you can use gpsd for interacting with your GPS. If you're on Windows using the new Location API in Win7 would be nice, but, more realistically, you'll be listening on the serial port your GPS uses and parsing NMEA packets. No worries, this isn't that hard to do. After a either a certain interval or time or distance traveled, send a few pings to a server and download a reference file. Dump your data into a CSV or use OGR to make a shapefile, KML, PostGIS layer, whatever.
http://www.sensorly.com/ Run it on your phone manually (or have it triggered intermittently). The 3G, EDGE and wifi coverage that your phone detects is uploaded to the central server and within a minute your phone receive the updated maps. You can only contribute to the maps for your phone's carrier, but you can view the maps for all carriers.
Not open source but many operators worldwide use TEMS. You could try your slashdot charm and see if they have evaluation or trial packages. Product was originally developed by Ericsson and later sold to Ascom I believe.
http://www.ascom.com/en/index/products-solutions/technology-platforms/platform/tems/solutionloader.htm
I'm a bit rusty, but since your network is probably TCP/IP, can't you do this with standard TCP/IP tools?
The only thing that could be a problem is measuring "signal strength". However, but a lot of mobile chipsets work under Linux. Even tools running under Windows, there may be access to this information (netstumbler)
For latency, you could just use ping:
Also, take a look at ping -f (use with caution) and traceroute.
For bandwidth measurement, setup an FTP server and script automated downloads from your clients using a .netrc file
http://www.stratigery.com/scripting.ftp.html
Or just use a command line HTTP client like wcget -- depending on your requirement.
You can use Gnuplot to chart data such as signal strength or error rates by location. For example, if you time sync GPS data and your statistics you can emit surface and contour maps (such as the second glass.dat example near the bottom).
There are far many more factors at play than signal strength. Your bandwidth and latency measurements are only slightly correlated to signal strength... much more to SNR and network load.
The fact of the matter is for EVDO (for example), network providers make bandwidth claims that sound impressive, but the number they state is the maximum air rate supported at one data burst per RF channel per sector. When you are the only active user on a base station sector/RF channel, you get a large proportion of the available data bursts. When others are online, you all share the channel so your modem gets a smaller proportion of the data bursts, even though the signal strength has not changed.
To complicate matters, CDMA networks do "soft handoff", so you can't just look at the signal strength from one tower, you have to compare SNR of several towers simultaneously. To do a real analysis, you will need system loading information and several specialized RF measurements. The carriers are not likely to provide you with the former, and the latter will cost you in gaining the expertise and test equipment.
In the end, the project was cancelled before we got a chance to get to test 3G coverage. But we did get to think about it. Our customer was a fleet of cargo ships, going through a fixed path on the Rio de la Plata (River Plate). Basically, we were going to install our CCTV system in there, and have it push images and other information to our servers whenever it had signal. We wanted to know approximately in what areas of the river we would have signal. We were going to base our system on the Vodafone mobile connect driver. It's a set of Python scripts. Of course, it communicates with the modem using simple AT commands. It's released under the GPL. It is capable of measuring signal, sending and receiving text messages, and other nice stuff (like, well, actually dialing and calling PPP to stablish the connection). We had it working with several Huawei devices, but I know it works with other brands too.
Our idea was to modify this scripts so that they would try to maintain a connection, auto-dial every time it disconnected, and log the signal at certain intervals to a MySQL DB. We were also going to run download tests all the time automatically. Since there was no chance we would go on the ships with the devices (the ships were cargo ships that transported and extracted sand, and there weren't very comfortable, not to mention their average trip was at least ~72 hs.), so we wanted to do all of this automatically. The devices would also inform their IP to a web service every time their IP changed, so we could SSH in the machine running this tests in case we needed to change something.
We were going to add a GPS to this system, that would also log its position at certain intervals, so that we could then generate a color-coded signal map.
I hope this helps. It's really fairly simple. I would be happy to provide you with source code, but we didn't get that far into the project as to produce actual source code, since the customer changed his mind due to budget restrictions real early. Feel free to contact me if you have other questions {almafuerte (at) gmail (dot) com}
WTF am I doing replying to an AC at 5 A.M on a Friday night?
http://www.cs.wisc.edu/~suman/pubs/citywide.pdf
ABSTRACT
We describe our experiences in building a city-wide infrastructure for wide-area wireless experimentation. Our infrastructure has two components — (i) a vehicular testbed consisting of wireless nodes, each equipped with both cellular (EV-DO) and WiFi interfaces, and mounted on city buses plying in Madison, Wisconsin, and (ii) a software platform to utilize these testbed nodes to continuously monitor and characterize performance of large scale wireless networks, such as city-wide mesh networks, unplanned deployments of WiFi hotspots, and cellular networks. Beyond our initial eorts in building and deploying this infrastructure, we have also utilized it to gain some initial understanding of the diversity of user experience in large-scale wireless networks, especially under various mobility scenarios. Since our vehicle-mounted testbed nodes have fairly deterministic mobility patterns, they provide us with much needed performance data on parameters such as RF coverage and available bandwidth, as well as quantify the impact of mobility on performance. We use our initial measurements from this testbed to showcase its ability to provide an ecient, low-cost, and robust method to monitor our target wireless networks. These initial measurements also highlight the challenges we face as we continue to expand this infrastructure. We discuss what these challenges are and how we intend to address them.
one possible solution:
Download the android SDK; write an app; run it in the emulator that comes with the SDK.
I'm not sure how much work it'd be to tie your 3G card(s) into the emulator (that comes with the SDK), but it's possible.
Linux would be my first choice, but the SDK also runs on windows or mac os.
Bonus for getting a useful app included in the app store.
While the goal is an admirable one, there are just too many variables to even begin to consider really doing this.
First off, you aren't working with a stable environment. There are changes being made to the cellular infrastructure all the time. Changes to settings in the towers, changes to the tower equipment and addition of new towers. You will not be informed about any of this making your information out of date almost immediately.
Next there is the problem that most throughput problems are going to be caused by overcommitment of resources, not lack of radio coverage. You will find that there are patterns to this, but again this sort of thing changes over time.
Essentially, what is wrong with the carrier coverage maps is they aren't detailed enough and do not account for throughput problems. You are going to have the same problem, only perhaps worse. In short, I would expect any data gathered in this manner to be extremely unreliable because you don't even know what variables you are missing.
If you are evaluating these cards for use in business, then you don't really care which network has the best coverage per se, you care about the effective coverage with the cards you have available to use with each provider. You also care more about the speed (throughput and latency), and continuity more than you care about the signal bars shown on the device (or even measured internally).
If you have an idea where the devices will typically be used, I suggest you pick a number of test sites (say .. 25 or 50) and go test each card at all of them. You can run ping tests for a few minutes and record the average card at each location. (Especially watch for dropped packets). Then you can run a speed test from something like speakeasy's speed test site. These sites could be indoors, outdoors, at the train station, wherever. Another thing that may be important (but it a little harder to test) is use under motion. For example, my friend has a WiMax modem. When he has coverage with 2 bars, his connection is faster than my HSPA modem with 4 bars. At the same time, I have coverage *everywhere*, whereas his isn't remotely usable on the move, and often not at some locations. You could simply do a non-scientific test by using each one on the same stretch of train track, and see which ones have problems and/or which ones seem more zippy. If you can't tell the difference, then it probably doesn't matter for practical purposes.
...or "Spirent UTS"
I used to do this kind of programming all the time for Qualcomm and Kyocera Wireless. Cell phones have a User Terminal Diagnostic Monitor running on them constantly that can be used to get almost all data in realtime out of the phone through it's data port. There are a couple of areas of memory that are write only for things like passwords, but you have access to pretty much everything else.
A bigger number on the same device probably means you have a better signal. A 'two' on one device could be equivalent to a 'five' on another or a 'one' on a third. There is no standard that is used across the industry, or even across all devices from a given manufacturer.
When a device manufacturer gets customer reviews that say "I only get one bar with your phone but two from company X" the device manufacturer can either try and explain repeatedly that their one bar is better than X's two bars and that unlike X you can still make phone calls on our device on one bar. Or they can just double the number of bars reported on the next model so they don't look worse than X.
Which do you think they do?
To do a real test you need to use a constant antenna and location, attenuating the signal gradually until each device stops functioning. The amount of attenuation it can take is a crude indicator of the quality of the radio.
We are going through a similar thing at work, where our field supervisors whom have vehicle mount PCs were having problems with connectivity in several locations. We devised a test, based on the speakeasy.net throughput test (ran 3 times for good measure) and a file download from microsoft to determine where good/bad locations were. Not very scientifically accurate but it works. We took the vehicles out, each with laptop and vehicle mount PC with diffrent configurations, parked them next to each other in several locations, ran the tests and got the results. We also kept a log of known bad locations, and some could be explained (canyon conditions in urban areas, hills, etc.) and others not so. Also user expectations is another ongoing problem. They expect wired eithernet experence through a cellular modem.
1) Ping Test. A simple ping to the nearest visible IP address, usually the GGSN, will give you the round trip time for a session in progress. You are looking for the variability. A simple ping -t command for a few minutes and plot the results in excel. You can vary the packet size, but the two tests below give a more accurate test for throughput. Protip: if you want to measure radio channel activation time too then you need to know a few network parameters, specifically the radio TBF timers. You would only do one ping at an interval greater than the timer so that each new ping needs a new radio resource allocation request. The round trip times for the first ping will always be longer, and recording variability in the allocation requests is useful if you are the network, not so much for users.
2) TCP Test. Use an FTP session download a file of known size to get the actual time taken. Then you need to upload that file back to a separate directory and compare the two (or just see if it opens). You actually have to do the math to get the bits per second, neither the windows or Linux native clients show it correctly. bps = bytes_recieved * 8 / seconds. Protip: you need to test several file sizes to get an accurate picture. I'd suggest one run would up/download several 300k and 1Mb files and one 10Mb file. To complete the test in the field you do need to check the upload is successful using a terminal session, but remember not to have the session open using the test equipment as it will bias the test.
3) UDP Test. For assessing the actual performance of the radio network itself the only test that has any relevance is a unidirectional UDP stream because the radio resources are allocated unidirectionally and have asymmetric speeds. TCP is not a test of the radio network itself as the uplink channel causes throttling on the downlink due to the slower uplink. You need a utility that sends UDP packets of a certain size, and a utility to receive them. Ideally the packets have sequentially numbered content because what you are looking for is out-of-sequence delivery. You measure the bandwidth by sending a known number of packets of a known size and simply subtract the timestamps from the client to get the duration. This has to be tested separately on the uplink and the downlink. To instigate the downlink stream you again need a remote session, ideally not using the test equipment.
I was a 3G data engineer for Sprint PCS during the launch of 2.5G and 3G data. There are a few problems with the type of testing it sounds like you want to do.
First, I would suggest reading the specs from the IEEE on CDMA 2000, aka 3G. CDMA2000 allows the ability to allocate and de-allocate bandwidth on demand and based upon quality of service configurations. At night, with no one on the cell tower, you're going to get the full pipe for data. You'll see bursts up in speed but then several things can happen. First, voice takes priority. So at midnight in this scenario, I pick up my phone and make a voice call and I take priority. You're pipe just got smaller. The next variable is overall tower usage. Cellular towers shrink and expand RF power with regards to usage. As more users get on, the cell tower will reduce it's footprint. So even if hardly anyone is on the phone, but there are a ton of subsribers on a cell, it can drop it's power. So your bandwidth and RF are variables which change by the second.
So if you're just doing a "Hey lets just see what we see," type of test, then expect a huge array of data with not too many descernable results. If you're looking for data to be compared with something else (carrier vs carrier, region vs region), then it also won't be terribly useful. As a geek, it'll be cool to know you can set the test up, but as a quantitative analysis tool it won't be repeatable or statistically useful. You also can't really compare it to desktop loading times either. The images you "download" via a wireless carrier are not the same. Go to Yahoo and download the GIF for their logo using an aircard or wireless carrier device. Now, download the same with a desktop PC over a wired (or other non-wireless carrier method). Not the same file size, huh? :)
Just as an aside, I used to test it using Stick Figure Death Theater ( www.SFDT.com ) since you can start a really long download and watch the stats. Also, it was entertaining to watch at the same time.
It is seldom worth the effort to try to "resolve" anything with any of the vendors. You just have to vote with your money and go elsewhere, when you can, when you have problems.
That said, I have an at&t 3G card and had good coverage at work, but not at home. My house is less than a mile from the tower that provides coverage. I am a avid ham radio operator and spent some time on a personal technical solution. I do microwave work and took time to match signal coverage to topo maps
and that is how I found the tower that is supposed to cover my house. I then confirmed that I live on a small mesa and the placement of the tower, had their
signal going over my house and leaving me in an RF shadow. Once the technical problem was clear, I bought two high gain directional antennas and the
cable needed to plug into my 3G card. I ran low loss feed line up to one of the antennas on a mast on the house, and bore sighted it in on the tower. I plug
the 3G card into the external antenna at home and that problem is solved. An external antenna on the car helps as well. If I need good coverage in an area
of weak signals, I put the second antenna on the 41' pneumatic mast on my ham radio van and aim it at the nearest tower. This is not what most people would
do. It works for me. at&t was, and never will be any help. My house showed up as well covered on the at&t maps, and I made it so. On to other tasks.
Try the application CellMapper from the Android market. It creates maps of user-submitted cell signal strength. Some of the areas you are interested in may be covered already, and any new data you submit will help to benefit the community as a whole.
I found it extermely easy to write a program myself for my windows mobile handset. It took me about an hour to write a program which a) collects network information (available techniques such as GPRS, EDGE or HSDPA) and signal strength along with tcp ping statistics (for some reason some Finnish wireless base stations block icmp ping) and b) gps data (time, speed, location). This gave me a possibility to gather quite interesting data about both coverages and the effect of speed as I ran the mapping program a couple times while traveling the same route by train. If you happen to have a wimo 6 -phone and have _any_ programming experience you should be able to hack something like that together (using free tools) very fast.
Shameless self-promotion ahead...
This is more a suggestion to help with mapping received signal strength (RSSI), rather than data network latency and bandwidth (you can argue that those data network metrics rely heavily upon your RSSI!):
Grab an old Nokia, use gammu to enable Network Monitor mode, fire up a GPS and display the combined information streams on a map. I did exactly that as an experiment using a Nokia 3310 and a Navman GPS receiver. Interesting to then correlate the signal peaks to the actual base station locations.
The main caveat is that the old Nokias in question only do (I believe) dual-band 2G GSM at best, so you won't actually be able to measure 3G W-CDMA statistics as if you were connected with a 3G data device. I would offer, however, that 3G RSSI might be related to that of 2G as many of the base stations handle both services (ignoring differences in signal propagation characteristics). A starting point at least...
Hook up a GPS device to a laptop, the 3g cards each to their own laptops, drive around while they ping common hosts and/or do other network tests and your gps maps out locations.
Warning: shameless plug dead ahead...
Look at http://www.p3-group.com/communications/en/home.html, it's their core business and they just opened an office in the US.
If your end users will need to run 24 hours a day you will need to test 24 hours a day.
We have number of little (Linux of course) boxes using 3G USB modems here in Sweden, and have discovered that Telia does all sorts of maintenance at night and at weekends.
Furthermore the latency (as measured by ping) varies quite a bit depending on time of day, i.e. when the local farmers are downloading their porn.
Another thing we've come across is that the versions of the USB modems changes with great frequency! Which of course means when they break you have to be very careful who changes them.
Good luck!
Sorry, nothing profound to put here! (http://www.abacus4.com/
There are plenty of expensive software solutions for this that are used at a professional level (ROMES, NEMO TEMS etc).
If you are after a free solution, the AT commands of most units Huewai, Option etc will give you network information,
For example: Signal as RSSi. (look under 3GPP TS 27.007) AT+CSQ? gives a number 0 (-113dBm or Less) to 31 (-51dBm or greater)
Failing that a lot of the dashboards have open APIs (in the UK Vodafones dashboard gives you access to lots of information)
Most of the Samsung mobiles have a diagnostic screen giving levels Scrambe Code and Cell IDs
For more info Siroda.co.uk or pm me
Si
You could try to use a Nokia Phone. Have a look at:
http://en.wikipedia.org/wiki/Nokia_network_monitor
Getting the data out of the phone might be done with some tools like gnokii. Don't know if modern 3G Nokias can be tricked into showing the monitor, but maybe the providers could help you out.
Back in 2001/2002 I was doing drive testing for US Cellular. One day we were cruising through some random suburb of northern Illinois. My driver that day was an Iranian American. Nice guy, but not the usual bleached white skin tones folks are used to seeing in such a location. Someone called us in to the police as possible terror suspects, driving the neighborhood around with "computer equipment".
So we got pulled over by a fleet of police. Took about 5 seconds for them to realise what we were doing and we went on our way.
As to the original question, we ran two phones, one that would continuously make short duration calls to test the ability to the network to identify and connect new call, and another that would try to stay online for as long as possible, testing the network's ability to hand off calls from one tower to the next.
If you are planning on testing a decent sized area, I would recommend using a similar system. If you're just going to go to specific hot spots and run a couple of tests at each location, I wouldn't go to such lengths though.
-Rick
"Most people in the U.S. wouldn't know they live in a tyrannical state if it walked up and grabbed their junk." - MyFirs
"In order to resolve an ongoing issue with a vendor..." Good luck with that. Doing the testing itself is very technically interesting and worth discussion. Trying to resolve an issue with a vendor by "proving" they have a certain level of service is a different matter entirely. That type of approach to resolving business disputes just isn't the right approach. It's a good example of trying to resolve business/legal issues with proving something in technology, and that is very difficult.
Just because I can hook a shark from a boat, I do no offer to wrestle it in the water.
@Ethanol_Fueled: Why are you offering advice based on your imaginations? Been drinking fuel?
Using only Linux, I connect via ATT wireless with Sierra & Option 3G modems. Last year I also used Hauwei modeon Alltel & VZW. In 2006 & 7 I used Sierra modems on VZW.
First connection with 3G modem & NetworkManager is easy. Click, click, click, choose GSM, enter "*99#", enter "isp.cingular", apply, OK & OK. No PWD, no UID, no proprietary software.
Even you could do it.
I'd mod you Insightful, but if this telco has any competitors, a nice heaping pile of public shame could motivate them to take care of the issue...Post the results of the "cellular wardrive" online and send links to the competitors, ideally to someone in marketing if you can find the contact info. Again if there are competitors, maybe you can get a SIM from them and record data from that device simultaneously so you can compare the signal quality. The company with the best signal quality in the area would be very interested in the results.
As for capturing the data, it shouldn't be too hard on any Linux device as long as you can access the signal quality of the cellular modem. I know you could do it with a Python script or maybe even a shell script on an N900. The hard part will be using the data to produce a map visualizing the signal quality in different areas.
"When information is power, privacy is freedom" - Jah-Wren Ryel
Mod this guy up. One of the few posts that a) knows what he's talking about b) can put the technical bits for a layperson.
Here's what you can do without the very 'spensive custom 'ware the big guyz use.
1. Go to tower site.
2. Back off a couple of blocks on the axis of the "Alpha" lobe antenna.
3. a. Take a reading of signal strength, log it.
3. b. FTP a file up and down, log time required.
3. c. Run Speedtest.net, log results.
3. d. Run Speakeasy.net/speedtest/ , log results.
4. Run out a half a mile or so on the "Alpha" axis, keep going until the carrier is -20db.
5. Repeat #3.
6. Go out as far as you can on the "Alpha" axis and still get one bar of signal. Log the location.
7. Back to the tower, find the "Beta" lobe, back off a couple of blocks, repeat steps 3-4-5-6.
8. Back to the tower, find the "Gamma" lobe (if there is one), back off a couple of blocks, repeat steps 3-4-5-6.
9. Repeat for every town in your area of concern.
These links may be helpful in locating towers. Remember towers are often shared.
http://www.antennasearch.com/
http://www.celltowerinfo.com/CellTowerLocationMapsIndex.htm
http://www.crowncastle.com/tower-rental-services/tower-locator.aspx
http://wireless2.fcc.gov/UlsApp/AsrSearch/asrRegistrationSearch.jsp
However, don't be surprised if you find towers absent or not listed. I reported one dedicated location as being 300' away from the location specified in the FCC license, and the reaction from the engineers led me to believe this was not uncommon.
And, as noted previously, don't be surprised if the sales droids give you the 'Mechanic's Shrug' and then redouble their efforts by inviting your bosses' boss out to the golf course.
There is a tool for MS Windows called Radio Mobile that uses topo maps to simulate radio coverage, it may help.
http://www.cplus.org/rmw/rme.html
I have used SMSlib for a similar purpose. It is an open source Java library for sending and receiving SMS with a wireless modem. I used it as a framework to communicate with the modem and extract its signal quality measurements. The basic algorithm is: -- Force the modem into 3G mode (that's what I'm interested in) -- Loop every minute: -- Read and record RSSI, BER, SNR You can read about the details on: http://knol.google.com/k/novak-petrovic/modem-based-3g-signal-analysis/1j9pv1wo3nq4h/20# There are also my measurements at two locations: "good" and "bad". With a bit more work this approach can be extended to correlate RF parameters with some other (IP) measurements you mention. It can also be extended to make it a mobile measurement that you require. I am currently working on a more accurate interpretation of some of the RF measurements. Best regards, Novak.