Filesystem Problems with the Treo 650s

Kaisa Tarasov writes "It turns out PalmOne's new Treo 650 is shipping with a major problem that's causing first adopter users and developers to cancel their orders in droves. The new Treo, along with the Tungsten T5, utilizes a new FAT based nonvolatile file system. Not only is the new system much slower, as the data has to be loaded into a SDRAM chip before running, but in this filesystem PalmOne switched from using directly addressable storage, to storage addressed in 512 Byte blocks. This has caused many files to swell in size - up to 500% in some cases (such as the address book). Users, already flustered with the small 23 MB of available memory, when trying to sync their old data onto the new device are discovering that their old data does not fit on the new Treo. What does PalmOne do?"

Comments

Ouch!

[Anonymous Coward]

Who gets fired for this? Q&A? The engineers? Managament?

It's too bad that such a glaring problem got missed in production. Hopefully they will be able to fix it.

I think PalmOne is right

[aldoman]

I think that PalmOne is right in choosing to use a block based filesystem. There is obvious limits on the the old method, and while this has some problems, from what I gather they could easily solve them by instead of having each contact data in a seperate file, moving it to one file (or having a 'zip folder' which could expand and look like a normal folder when opened).

The main problem is that PalmOS is looking very dated compared to WinCE and Linux, and it's going to require serious pain that I don't think PalmOne can take to modernize it fully. This is just one step.. think how much it's going to hurt to get proper multitasking in etc...

Re:I think PalmOne is right

[rudedog]

I think that PalmOne is right in choosing to use a block based filesystem.

Except that they haven't really. They've moved from storing their databases on battery-backed RAM to NVRAM. Their implementation uses a block-based filesystem, but the API continues to be the same as it always was (DmQueryRecord, DmWrite, DmReleaseRecord, etc.).

The backing store uses FAT, but I believe that each database is still stored in a single FAT file, that the programmer never sees or knows about.

PalmOS uses a cache to arbitrate between the NVRAM backing store and the Dm* functions. For performance, their cache implementation pads records up to the nearest 512B block, which is why databases with small-sized records seem to bloat.

The solution to me is simple: add a new header flag to the database that tells PalmOs not to pad records on that database. This would go back to the way that PalmOS exports databases to normal flat files without padding each record.

ARGH

[Andy Dodd]

Dammit, yet another possible replacement for my Kyocera 6035 proves to be insufficient.

I was hoping for the 7135 to drop in price, but Verizon outright pulled it instead.

None of the current batch of smartphones appeal to me in design. They're all more PDA than phone, the Kyos were EXCELLENT phones. I *need* tactile feedback when dialing my phone, and all of the current smartphones use on-screen dialing.

Re:ARGH

[Doc Ruby]

I went from a Kyocera 6035 to a Treo 600, and I've been extremely pleased - smaller, but not too small (like many phones), smarter, but not too smart (like a palmtop PC with all its problems), acceptably good Internet connection, color, stereo music, 1GB SD cards, camera, keyboard + stylus, yeah! I might not go for a 650 so fast, since they're delaying PalmOS6 (multitasking), and skipped the 1.3Mpxl camera (though the new VGA camera seems much better). But this FAT issue seems fairly trivial, especially with 1GB+ SD cards and Bluetooth. Maybe the next iteration sometime in 2005 will have all that, plus the hirez camera, plus EV-DO/EDGE WAN (>130Kbps, up to 1.5Mbps) which is the threshold for the mobile multimedia terminal the Treo 600 almost became.

Frankly, I chucked my 6035 beneath the wheels of an oncoming train to stop it (the phone, not the train :). Its many bugs and inconsistencies made using it like shaving with a nicked razor. Treo 600 reinspired my love of Palm - once again, Pilot is my co-god!

Palm Reach Out to The F/OSS for Help?

[Levendis47]

Call me wacked but sometimes the best way to wipe egg (or in this case, a whole omlette) off your face is to ask someone to wipe for you (eek...).

Palm could reach out to the OSS community for help in dealing with this...

1) Rapidly turn around a six-month trial developers kit and a limited-licensed SDK for OS development.
2) Make it extremely easy to find/download/bootstrap.
3) Setup a contest... List the top five major issues/flaws in the software at any given moment with corresponding prizes for the individual/team that develops a viable solution for a given issue/flaw.
4) Filter solution entries though a rapid in-house QA and system testing process.
5) Release patches in "leap frog" pattern (i.e. say four-month cycles overlapping for bi-monthly update releases).
6) Build and distribute a Palm Desktop conduit for System and Application updates. Call in "pa1m OneUpdate Utilities" or such.

Just an idea... Run with it at will...

I have a Treo 600 that I waited for two update cycles to occur before I bought... I've been burnt by Palm and WinCE before. And while I loved Handspring products, I can't think of a single one that didn't have some odd problem (shiver, the Visor Edge...).

cheers,
Levendis47

Perhaps it makes some things easier...

[SuperKendall]

One advantage I could see is that the FAT filesystem is well understood and supported by a lot of things - it might make it much easier to mount the device as portable storage and make direct modifications.

Re:Treo 650 Scam on eBay

[bumbobway]

Here's an example: http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&cate gory=38331&item=5732298193&rd=1

reiserfs

[wotevah]

They should have licensed reiserfs. It uses a block system but small files can share a block:

http://www.namesys.com/v4/v4.html#sharing_blocks.

You can get a special license to include it in your own proprietary OS.

Palm OS vs. Copland

[TimmyDee]

As I've been monitoring the discussions of the 650 at TreoCentral (I'm thinking about getting one myself when the GSM version comes out), I couldn't help but thinking that PalmSource and PalmOne seem to be in a position very similar to Apple a few years ago. I know, they're two separate companies where Apple was (and is) one and their new OS is actually off the ground, but bear with me.

A few years ago, Palm/PalmSource probably realized that their OS wasn't going to cut it in the New World of modern computing. They were making the transition from 68k processors to the StrongARM/Xscale series much like Apple made the switch from 68k to PowerPC. All arguments aside, I'd say this was the right thing to do for both companies, but it left them in a bit of a predicament -- legacy code. The only option for both companies was to develop an emulation system so the old could be run on the new. They both work really quite well, but everyone knows you can't run on a hack forever. The time to break with the old had come.

So, Palm decided to start developing Cobalt and Apple started to develop Copland. Preemptive multitasking, protected memory, better multimedia handling -- the calls to arms were the same. Yet where Apple failed with Copland, Palm didn't. Sort of.

Copland was a nightmare. Years of legacy code had turned the Mac OS into a bunch of spaghetti and for some reason the Copland developers thought they could use that spaghetti and bake a tieramisu. It didn't work. Drained of billions of dollars sunk into development, Apple started shopping around in 1996. They looked at BeOS (to what degree of seriousness is a matter of debate) and NeXT and some others, thankfully settling on NeXT. Palm, too, had likely started from the bottom up, found themselves a bit stuck, and then stumbled across the devalued Be, Inc. Purchasing Be, they gained huge strides in the multimedia area and were on their way. They also created PACE, an emulation environment similar to Classic in our beloved Mac OS X, for all that legacy code.

Cobalt should be a runaway success like Mac OS X is. But it's not. You could say that Cobalt is like Mac OS X when it was new. Everybody thought it had great promise, but even Apple was afraid to use it because it just wasn't finished. Now, I'm not sure how "unfinished" Cobalt is at this point, but it could be in the same boat. There are also issues of licensing fees (which I hear are significantly higher for Cobalt compared to Garnet) that cause the analogy to break down a bit, but for the most part it holds.

So in the end, Palm OS 5 is starting to look a lot like Mac OS 9. It works well, but man does it have its problems. Adoption of Cobalt will be key, but PalmSource needs something killer to drive that. It's a shame for PalmSource/PalmOne that they didn't pick up Dominic Giampaolo with the Be acquisition, but I'm also a Mac user and I'm sure glad he's on our team now.

Amen

[Anonymous Coward]

Palm has been underwhelming ever since the IIIc.

They're caught in this terrible "Do I want to be an organizer or a pocket computer" conundrum, and they're starting to do neither very well.

So we have PocketPC which sucks or we go with Palm which produces sucky hardware (in 2004, they have no WiFi solution. Welcome to 1997, Palm).

Between MS and Palm, they'll kill the portable computer market for sure.

Will break existing applications

[iamacat]

On regular Palm devices, you can read from and write to database records directly. It goes something like this:

MemHandle mh = DmQueryRecord(db, recNum);
void *p = MemHandleLock(mh);
MemSemaphoreReserve(true); // write-unprotect storage memory
// Do some access to database record here
MemSemaphoreRelease(true); // Restore protection

Granted, MemSemaphore calls are undocumented and Palm asks you to use DmWrite to update a database block instead. The trouble is, Palm devices used to have 36K(!) of regular heap and for recent ones it's around 256K. And C++ compiler wants like 30K for each program/shared library (which is another sorry tale) for virtual functions, exceptions and jumps between 32K segments that you need to partition your code into. Finally, say your database record is a list of stuff >36K and you want to sort it. Imagine how good 2 DmWrite calls to do every record exchange will be for your performance and code readability.

So if you want to do some good stuff in your program, you just allocate "database" pointers and use them as your regular heap. I doubt it would for with Flash on Treo 650, since it will not even know which records are dirty. Even if they still support these calls, performance of your heap being swapped out to flash in 512 byte chunks would be dreadful.

The trouble is, programs that needed to use MemSemaphore calls are probably the ones that do something worthwhile. Try business applications, 3D games, VM-based programming languages... They are going to cripple the most cool programs written for their platform. Should have just included a rechargable backup battery just enough to swap out RAM on power failure.

It should be managers

[jinushaun]

They ultimately approve the specifications, so they're responsible for why the filesystem is the way it is.

the real story behind this

[Anonymous Coward]

I'm a third-party Palm developer, so let me shed some light on why this has happened. Or at least on my theory as to why it has happened.

OK, first of all, the current shipping version of Palm OS is 5.x. PalmSource (the software company that makes Palm OS -- Palm was recently split into PalmSource for software and PalmOne for hardware) has been working on OS 6.x. But, OS 6 is not out yet. They're basically done with it, but no devices running it have yet been released.

Now, OS 6 adds many great things (like threads, security, and graphics tools), but one of the big features is the ability to run everything out of flash (with a RAM cache). This is supposed to be a great feature for cell phones since people are always running their cell phones until the battery is dead, dead, dead.

BUT, there have been big delays with OS 6. So, my theory is that the T5 was originally intended to be an OS 6 device. But OS 6 wasn't ready, so PalmOne decided to just stick with OS 5 for it. And the thing is, they had already designed a flash-based device (the T5) but OS 5 doesn't have this flash-based feature, only OS 6.

So what did they do? They (PalmOne, the hardware company) added their own support for flash-based storage heap to OS 5. And they did it hastily, and it kinda sucks. Although it does basically work, but it's just not a good design because it's wasteful and stupid. (They probably didn't want to put in lots of engineering effort since they know it's only a stopgap thing anyway.)

Perhaps PalmOne will redeem themselves by making OS 6 available for the T5. Then it will have a proper implementation of the flash-based thing instead of this temporary crappy implementation that exists right now.

(I can't say definitively, but there are reasons to believe OS 6 does the flash-based thing by using the actual MMU. OS 5 almost certainly does it all in software (by shoehorning it into the system calls), which makes for a bad implementation because you can't really determine which pages are least-recently accessed to get good performance. If you do an actual paging system like I think they've done in OS 6, then you could easily pack small records into a 512-byte block of the backing filesystem instead of just mapping every single record to a list of blocks.)

So, to recap: T5 designed by PalmOne with OS 6 in mind; OS 6 (by PalmSource) delayed too much; PalmOne needs to release T5 so decides to go with (older) OS 5; OS 5 doesn't have flash feature necessary to even WORK on T5 hardware, so PalmOne has to add it at the last minute; OS 6 flash feature could/should be MUCH better.