Slashdot Mirror
Sending Data In Bursts of SMS Messages
An anonymous reader writes "Canadian carrier Rogers has been experiencing some extreme loads of late, as researchers at the University of Waterloo investigate the potential for sending data spread across bursts of hundreds of text messages. They sent around 80,000 messages in the course of a project testing a new protocol able to cram 32KB into 250 messages sent from a BlackBerry, reaching a rate of 20 bytes per second. The group thinks its protocol could be useful in rural areas of the developing world where text messaging is the only affordable, reliable link."
Don't give AT&T any ideas, you jackasses!
Make sure you get the "unlimited" text messaging plan before trying this...
so now will they bill $1 per txt each way?
and I'm in a major US city. it sucks when it's commonplace to get text messages out of order. Sometimes I'll get one that was sent several hours earlier.
...and got to feel the thrill of competition again.
Is it just my observation, or are there way too many stupid people in the world?
20 bytes a second? As in an SMS every 8 seconds? I can't think of a single situation I would consider this worthwhile. If it's a word document, why not send the plain text? If it's a JPEG, call the person and describe what you see. If it's anything else, drive to Starbucks for free wifi.
Sometimes when I get home from work my wife will be watching Judge Judy. When she tells me "there's nothing else on," it seems the best solution would be to turn the TV off.
Whale
If I was a cell phone company I would throttle anybody that tried something like this. Like one SMS per 5 seconds or even longer.
The SMS system isn't designed for 10,000 people all sending tens of thousands of messages per hour or whatever.
You pay: Monthly for a cellular package with unlimited texting
You get: 20 baud
Alexander Peter Kristopeit bought his basement from his mommy for one dollar.
In a completely unrelated story, the University of Waterloo has an unexpected ~$16,000 shortfall this quarter.
I judt got a nre Kinesis keybiartf so please excusr ant egregiou typos.
Text messages are one of the most awful forms of data on the cell network. On a 3G type network, they are just data, so hey, if you can do TXT on 3G, just do data. So what?
But on older networks, such as the proposed usage, they take up CONTROL channel space, and too much SMS is a DOS attack!
See Exploiting Open Functionality in SMS-Capable Cellular Networks:
Test your net with Netalyzr
They couldn't have built their own network and emulated phones to test this protocol, they had to go live with their phone provider? Some University. I bet MIT is laughing out loud.
Also, how's the coverage out there?
Being a massive cheapskate, I'm still behind the times and don't pay for data (yet), despite having a smart phone and having unlimited texts. I have to admit, I've wished something like this existed several times
Anyone care to describe why they couldn't just use airtime minutes and an acoustically coupled modem? Looking it up on Wiki, in general they were able to transfer 300 bps instead of 160.
More Twoson than Cupertino
Let's see, at 20 cents a message that test only cost them $16,000 worth of messages! And they managed to move all of 10MB... If my math is right. They should just spring for the pay-as-you-go data plan at the bargain basement cost of $1.99 a MB, they would cut the "cost" down to 20 bucks!
Are rural, developing countries really selling unlimited txting plans for affordable rates? If so, why is it that we let carriers in the developed world get away with robbing us blind?
A company called Jamun (http://jaamun.in/) already offers a email over SMS service in India.
Not trying to troll, but this is the wrong 'solution' for so many reasons. If SMS's can make the connection, so can other forms of packet radio.
I don't get the point of this. If you have a cell connection to send SMS, you already have SOME form of data connection. Even on a typical 2G network you can get 60-100kbps data. Are they trying to make this work on some old school network that no one in the US is using anymore? On top of that, Verizon had great rural coverage, 3G even in a lot of places. The only thing I can see that is useful, would be some sort of cleaver phone to phone direct transfer. But then again, if you are already on a data capable network... why??
Since when were text messages affordable or reliable? I thought delivery was not guaranteed. And as for affordable...not so long as the cell phone companies have anything to say about it!
Not even a real challenge.
Take the available character set; use that as your base (like base64/base92 to send binary data as clear text); toss in forward error correction and a you could do TCP/IP over SMS if you wanted to.
Seems to me a voice connection would provide faster data transfer rates than sending text messages.
... next year's April 1 RFC -- "IP over SMS Carrier".
Why not use yEnc (no pun intended)? It's been working for years on usenet.
So they essentially rediscovered WAP? Great work...
Also:
http://conversations.nokia.com/2008/11/05/nokia-life-tools-opens-side-door-to-the-internet-in-rural-india-and-beyond-via-sms/
http://conversations.nokia.com/2009/11/19/nokia-life-tools-a-life-changing-service/
http://conversations.nokia.com/2010/05/07/ovi-life-tools-lands-in-china/
(and related on above pages)
One that hath name thou can not otter
Speaking on behalf of the interests of the RIAA & MPAA, it is clear to us that this "new" protocol will be used only for the piracy of copyrighted materials. Sure, downloading a DVD using this protocol might seem like harmless way to pass the summer months, but the damage to our industry is incalculable. And although we are headed for another record year, we calculate that this has clearly cost us over $10 billion dollars in losses and must be stopped.
Sincerely,
Jack Valenti
(Yes, I know I'm dead. Want to make something of it?)
"Kittens give Morbo gas!"
This is good news (everyone), by the time you have torrented your bluray rip, it will be out of copyright.
Or not.
They whose government reduces their essential liberties for temporary security, receive neither liberty nor security.
I'll mostly leave it to others to enumerate the many flaws in this, except to note that under AT&T I often had text messages arrive hours or days late, or never. But I do have to applaud this group. This is, by a wide margin, the worst idea I have ever seen in a /. story. Are we sure this wasn't a belated April Fool's gag?
> The group think their protocol could be useful in rural areas of the developing world where text messaging is the only affordable, reliable link
It's a fun little project, but in what circumstance would this *ever* be the best use of a mobile network? If you've got the signal for SMS then you should be able to also at least use a voice call to transmit data (not sure what the max would be, 14.4kbps? 9.6kbps?) if not full GPRS (56-114 kbps). 160bps is not very impressive
If I'm doing my math right:
160 character limited frame
Just using "a-zA-Z0-9" gives us 62 characters - throw in a few punctuation for 64 which we can use for base64/MIME encoding. Giving us 6 bits to use per character(byte)
Assuming an 8-bit byte in original data, 33% larger
160*6/8 = 120 bytes, 8 bits each in each SMS
250 messages of 120 uncompressed bytes each is 30 KB or 240 Kb
If I remember SMS already has sequencing in its protocol so you shouldn't have to sacrifice your own bits for that. SMS has a broader set of characters than 64 so we can inflate that number. What's the big deal?
from 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0
to 45 2F 6E 40 3C DF 10 71 4E 41 DF AA 25 7D 31 3F
Maximum carrying load of a Yak: 70kg
Weight of a 32GB micro sd card. 0.5g
Having your own 3rd world petabit network: priceless.
The old WAP1 protocol had the ability to use SMS as a bearer.
This was thrown out and done away with. Its slow, ineffective, and fucks up the control channel.
P.S. this is still in use for WAP Push on many networks (like you know, for MMS notifications). Good way to invent nothing while fucking up a production network guys!
A single low-end jury-rigged SMSC is well capable of over 5K TPS. 80K messages won't even break sweat on any telco's network.
That said, it's a pretty useless medium of communicating any significant amount of data. GPRS or even WAP are much more efficient and capable of dialup speeds. And hey, developing worlds have much better telecom networks than these kind of "for developing worlds" stories give credit for. At least in India, SMS is essentially free (costing less than $0.0001 (yes not a typo!) per SMS in volumes of a thousand.
- mritunjai
I'm not one to stand in the way of research - but 20 bytes per second? I'm sure they should be able to design some sort of adapter for 300bps modems and use those over the cell phones as voice signals instead, and have a substantial gain in transmission speed...
Now considering that most cell phone carriers world-wide actually charge a fee nowadays for SMS messages, ESPECIALLY in underdeveloped countries, sending a whole lot of SMS messages is probably not going to be more economically viable than hiring Jose (or Ahmed or Abayomi) for a dollar to take a CD to the other town and back on his bicycle.
There's a certain type of mind that tries to innovate by brute force, and yet while it's true that if you put a pair of wings on an internal combustion engine you are close to developing something new, you also have to remember to lose the 4 wheels on 2 axles, the transmission, the heavy steel body and chassis, and add a propeller...
Seven puppies were harmed during the making of this post.
This sounds like Pigeon-IP without the feathers.
I have to pay $10 when I text "HAITI" to 90999! I thought Microsoft was paying.
It was submitted as April 1st gag. Using itself, of course.
Only it's not too efficient, you know...
http://opencm3.net, http://www.nongnu.org/gm2/
About 25 years ago, TCP/IP experimenters on BITNET were sending IP packets as RSCS messages, which were limited to the same scale of data as SMS messages. It was slow as hell, but just like the SMS network, the RSCS network prioritized these short messages above other traffic.
This is the same network facility that inspired the IBM Reseach folks who moved to AOL to create the buddy list and everything that arose from there.
Funny how things come around over and over in the computing world - it's like nobody studies any prior work.
if someone were to actually do this, how many minutes would it take for their IMEI and SIM to be permabanned from the afflicted carrier
Snowden and Manning are heroes.
Texting used to be the only reliable link.
OK, I just don't get something.
GPRS would be a logical choice for data transmission on a barebones GSM network. Assuming that those "rural" providers absolutely don't offer affordable GPRS, you can use circuit-switched connections and just send our own data instead of GSM-compressed voice. IIRC, most random bit patterns are valid GSM packets (in all variants of compression), so that shouldn't be a problem. I presume one could even encode the data such that it could survive decoding, as long as you have a digital channel (say mobile to ISDN or mobile to T1 connections).
Of course probably you can't easily do it from a Blackberry. But if you're a telecommunications researcher worth your salt, you can easily get access to a software GSM stack, and inject arbitrary data into voice circuits, instead of GSM-compressed packets. This lets you have 6.5kbit/s or 13kbit/s depending on base station's utilization.
Those are, of course, raw numbers. With protocol overheads, this should be something like 5/10kbit/s, one would hope. Presumably to utilize the link in a best way, the TCP/IP connections would be re-terminated at both ends, and data re-packed in some custom protocol.
So using SMS for all this? It's the approach of least resistance, something you could do on a weekend as a proof of concept for your inquisitive kid maybe, but nothing more. BOO to the "reasearcher".
A successful API design takes a mixture of software design and pedagogy.
Really? Do these people not have a postal service? Per unit data a stamp is many orders of magnitude less expensive for sending data than a text message.
Damn_registrars has no butt-hole. Damn_registrars has no use for a butt-hole.
the data for a text message is sent through the control channels on the gsm system that otherwise represents unused bandwidth. this is basically free to the cellco's. it is also the reason for the small size of the payload
i have hooked up a motorola phone through usb to use as a mass sms system and can say that 8 seconds per message is the fastest i could ever send a message.
a text message is more reliable than other forms of data transfer on a cell. a message can be sent when a voice call is impossible because of a weak signal although at a slower rate.
having a protocol to be able to send large files through sms is a good thing.
the largest drawback to this scheme is the fact that cellcos charge extreme rates for sms data. it is a major cash cow for them.
I agree. SMS MEANS BY NO MEANS guaranteed delivery.
It works almost like e-mail and generally carries low priority.
We use SMS in our login procedure and we lose lots of messages. Or, they are delayed. Around 2%, i guess.
But I also do understand were they are coming from. Most systems actually don't need to send that much data.
For example, two b2b-systems could use SMS to place orders and stuff like that.
A remote repair shop or business could order parts and other goods through SMS, for example.
It could be used to send encrypted data between hospitals.
But it feels like there must be better options.
If I remember correctly, there are absolutely no guarantees an SMS will ever arrive (maybe something like within 6 months, but that is practically the same). Seriously. It's traffic secondary to the voice/data traffic. If there is too much SMS traffic, it will saturate the control channel, and the carrier will simply discard it. In developing countries this could potentially bring down cellular networks entirely, if the hardware can't cope with the sudden increase in traffic, rendering people not just data-less, but phone-less too. Ouch.
Never is, of course, a serious issue; but hours or days late would be solvable with the right protocol.
Bittorrent, in effect, deals with rather similar issues(since it is typically used to transfer files so large that they make common home internet connections feel like ghastly retro shit) reasonably effectively. It may take a while; but sufficient patience will get you past any number of corrupted blocks, dropped packets, hosts that disconnect, etc.
Any sort of latency-sensitive application will be right out the window; but dumping blocks of data from point A to ghastly-end-of-the-earth B should be totally doable....
Affordable and reliable, that is, until people start flooding the system with millions of text messages carrying data traffic -- at which time it will become suddenly less reliable, followed shortly by it becoming less affordable even for its intended use.
It is totally insane, but given the expense of having a data plan and the usefulness of even tiny amounts of data it actually makes some amount of sense to contemplate routing data over the SMS channel. It's no use for anything real time, but if you are doing things like constantly reporting data (latitude, etc.), sending or downloading things in the background (eg: email) then it might be possible to have no data connection at all and still get significant use out of certain types of connectivity.
Also, when not on a capped data plan (pay per MB) my carrier has a 100kB minimum for data - so for a 20 byte message I pay for 100kB. When I need to send a tiny amount of data it would make a lot of sense to send it over SMS.
What would happen if they sent that massive test to the wrong number? Bad NEWS bears!!
You are kidding, right?
---- Booth was a patriot ----
I tried to send a small mp3 but my thumbs got too tired.
Remember WAP?
The WAP service had three posible bearers, GPRS (the best for it), a circuit switched dedicated 9600bps link (later upgraded to 14.4kbps, or even 56kbps), or SMS.
Well yes, in WAP times there was a full spec on how to transport data on lowly SMS. As other posters have said, using SMS as a bearer for other data services is painfull, slow, ackward, and not such a good idea.
Ah, this brings memories!
http://www.m-indya.com/wap/wap_bearers.htm
*** Suerte a todos y Feliz dia!
Actually, it was an April Fools story.
They just used the described means to submit it, and it just now finished.
God is the universe (or more?), or as the Hindus call their Supreme God Head, Brahma: literally "Reality."
I'm not religious but some can be learnt, for good or ill, from the various faiths of the world.
Anyway, their is seriously only one entity -- Reality -- over which we and everything else are more or less a distribution of matter and energy (or who knows what more).
You probably think that's all poppycock, but I wondered...
Sometimes it's not about being a good idea. Sometimes it's just about seeing if it's possible. Lots of things are like this... like using a fishing pole for flying your kite, or building a makeshift jet engine out of a turbocharger. It's not necessarily practical, but it can be a bit of fun?
I have no problem with your religion until you decide it's reason to deprive others of the truth.
Hey that's great. If the dumb kids at Waterloo are excited about 20 bytes / sec, I have this here bleeding-edge V32 modem that'll do over one thousand bytes per second over a plain old telephone line. You can't imagine all the fun I had downloading JPGs on this thing back in the 80's^H^H^H^Hfuture telecommunications lab. Mmmm.. V32, that's like four mustang engines on a modem.
-Billco, Fnarg.com
There seems to be a disconnect between the Western Thinking and Eastern Reality. Take India for example, mobile comn is not really a luxury and even the lowest of the economic bracket posses mobiles. The fact is stranger than fiction.
SMS = Re 1 = approx 2 cents
GPRS/EDGE/2G Data for non plan subscribers = Re 0.1 for 10Kb that is ===> 0.2 Cents for 100Kb !!!!!
(its cheaper if you are on a plan)
why would anyone send data using SMS?
With Major Operators in Africa being bought out by Indian Telecom Companies... MTN etc
same will be the case with Africa..
Do you do a reality check ...or its assumed economics for you
I work with a non-profit mobile health company that uses a variety of technologies (SMS included) to get clinical data from health workers in remote villages into electronic medical records in central clinics. Our software uses GPRS/EDGE when available, but most of the time has to default to SMS because of coverage issues. Yes, sometimes text messages get lost, and yes sometimes they take hours (days even) to get delivered, but sending data over SMS is not a horrible idea (at least in this context). We've just learned that you can't send time-sensitive info over SMS (always call instead). We don't use acoustically coupled modems because that would be additional hardware that we have to purchase for our end users and the theft rates are decent for the Razrs we purchase secondhand anyway... The cost of a phone and an unlimited texting plan is less than that of a computer, a phone, an acoustically coupled modem, and a voice plan.
Our systems serve hundreds of thousands of patients across all of Africa and parts of South America. Admittedly, we don't use 'bursts of hundreds' of text messages (max:12 at a time) to browse the 'net, but the concept is largely the same. And it works. Every day.
I'm pretty sure I've written this protocol before myself, it was for uploading files to an old mainframe. (well actually a simulation running on a PC, but it worked the same.) All sorts of nasty limitations meant the file was first converted to ASCII then the lines were sent. They'd get miss ordered and misplaced but the protocol would sort them and retransmit the losses.
The Real News is that the Phone company is screwing everybody again. ..?
You know 80000 messages could have been just one small test file, how much should it cost to transmit a small file
I thought SMS was sent on essentially out of band data (or perhaps the control channel). This implies that there is some data channel already in place with a MUCH higher bandwidth. Why would anyone use SMS in this case ?
Summary states the students think it could be the only reliable form of communication in rural areas..what, transmitting data via SMS which is a (relative to 20bytes per second) high speed data protocol anyway. Interlacing their text data between Voice and other SMS -- sure, this sounds like worthy research *sarcasm*.
we have done this over Iridium satellite connections - SMS is one of the best messaging schemes for Iridium - works really well if you do a rateless erasure code on the data first !
Text messaging should really be nearly free. It can't possibly use as much overhead as a voice call...yet...I am intrigued to hear the suggestion that it is anything but obscenely expensive somewhere in the world. RAID was at one time an acronym for Redundant Array of Inexpensive Disks...but I guess someone found that an embarrassing affront to any intelligent being...yet it keeps happening. Human intelligence being insulted...that is.
In the late 1980's a program was developed to encode binary data in text format, called uuencode. It was followed by a shell script to take a filename, create an email message (or newsgroup posting) using uuencode, and package it as part of a shell script which could be fed back into a unix shell and would recreate the original binary file. I wrote a higher performance version in C called shar2 around 1990, which did some detection of file type, and optional preservation of modification. That program automated breaking the output into many parts of a given maximum size for transmission through limited message size channels. So using a bunch of SMS messages is hardly new.
Historical note
That appeared in comp.source.unix and was widely used. I maintained for several years, and stopped due to a lack of bugs and to avoid "creeping featurism" adding of bells and whistles. At a later date, someone else took my source, did some changes (perhaps 20% of the code changed), called it shar3, and released it, with my name pretty well removed and their name as author. Then they submitted it as "gnu shar" where it appears still, with only an obscure mention that I also had a shar program, the last time I looked.