Seeking Multi-Platform I/O Libraries?

An Anonymous Coward asks: "I'm just getting ready to plunge into a new project, and joy of joys have been given complete freedom when it comes to the implementation language - so long as the program will build and run on both x86 Linux and Windows. Now, I don't need a GUI, this is systems stuff only (processing binary executables in fact, so lots of bitfiddling and big nasty algorithms over hairy data structures) so pretty much all I need are standard IO libraries. C is currently at the top of my list..but what other language should I be looking at? I'm happy to learn a new one, and have the go ahead to do it..like I say, they want absolute speed. Can someone suggest a better language? C++ is out, it does come with a speed hit (using C++ properly anyway, not as a souped-up C). If I'm gonna take the speed hit, I may as well consider something like Ocaml which might let me claw the speed back with better algorithms and data structures.."

Comments

-1, Flamebait

[JabberWokky]

C++ is out, it does come with a speed hit

[ in my best announcer voice ]:

Let's get ready to RUMBLE!!!

--
Evan

Consider Python... Wait! Don't leave!!!

[Phoukka]

I know it's a bit of a stretch, but consider Python. Prototype the heck out of the system in Python, profile the application, then recode the bottlenecks in C. Use SWIG to generate your interfaces. Easier to program, easier to extend, easier to read/maintain. Shorter programming time, too.

You'll be happier, your fellow programmers will be happier, your successor programmers will be happier, and the chewy parts of your code will still be really fast. Think about it.

Re:Consider Python... Wait! Don't leave!!!

[Anonymous Coward]

Yes and no. Python is a nice language, and potentially useful. However, where I work we have a "legacy" system written in python with SWIG (interacting with C++).

The problem is that under large workloads (which is normal for us) you end up with python spending more time marshalling and unmarshalling objects. It's a PITA. I blame this mostly on SWIG (which I am NOT a fan of. Don't get me started on what the maintainers consider good development practice.)

Python's a great choice if you can do it all natively. It's also a great language to prototype in and then "translate" to another language like C++ or C or Java. (depending on task and preference.) But I wouldn't do the python+swig thing.

[Note: I'm only posting anonymously to protect my identity. There are certain political factions at work that read /. and would be very unhappy with me discussing this.]

Yes, try O'Caml!

[Tom7]

Yes, I would really recommend O'Caml. Here's why:

If you just write the same program you would have written in C, the speed will be quite good, probably about 20% CPU-slower than C. (And if your program is IO-heavy, you might not notice this at all.)

If you have any sort of limited time or interest (as most projects do), you'll be able to write a much better program in O'Caml than you would in C, because:

- Because it's safe, you won't need to ever spend time tracking down or debugging core dumps or memory leaks. Because it's statically typed, a large percentage of bugs are caught at compile-time.
- If your program is interacting with the network, you won't need to worry about buffer overflows, format string bugs, or most of the common security problems.
- O'Caml has a much richer core language than C, with support for algebraic datatypes, pattern matching, higher-order functions, threads, modules, and objects. You can do a lot of great stuff with these.
- O'Caml has a nicer (though not as nice as, say, SML) module system, which keeps your program from getting unmanageable, and helps isolate faults to a particular module.

And by better, I also mean faster -- development wisdom says that algorithms and data structures are what matter most, not just the instruction-level efficiency of your code.

Of course, if you don't know the language, then it will have a higher startup cost for you. But I think it's worth it; you'll learn a different programming style that can help you think in new ways even when you're writing code in Old School languages. =)

c++ is out?

[Aniquel]

I'm really very curious why you decided that c++ is out. I understand that the common (mis)perception is that c++ is slower - but let me ask this: Have you ever benchmarked it? If not, then I strongly suggest that you don't discount c++ out of hand. It has the cross-platform io facility of which you speak (streams), already has all the (completely debugged) algorithms and advanced data structures. Look, nothing is going to be faster than c (except for hand-tuned assembly) - If you absolutely need every little bit of performance, then don't bother with a language other than c. But, if you're looking for a language nearly as fast, with a complete template and streams library, that's portable, then you ought to seriously consider c++. (btw, I've written extensive projects in c++ (25000+ lines) - There isn't much performance difference, and the benefits to using it far outweigh any other penalties.)

Re:c++ is out?

[jmv]

My experience shows that in many situations, C++ can actually be much faster than C (not always of course). The reason: templates and inlining. With inlining, not only do you save function calls (which usually aren't that expensive), but the optimizer is free to use common sub-expression elimination across the "call". With templates, you can produce better generic code. Just compare the C qsort to the C++ sort algorithm. In the first case, you go through a function call by pointer (for the comparison operator) which is *very* expensive, while in the second case, the function will be optimized just for the type you need.

Re:c++ is out?

[gkatsi]

Even though you have it right that it is a misconception that C++ is slower than C, you miss one very importatnt point: the supposedly slower features of C++ (like virtual functions) do not have an equivalent in C. In fact, in order to achieve the same functionality in C, you will have to hand code what the compiler already does for you in C++. But we already know that compilers are better than humans in avoiding errors and applying the same solution over and over with good efficiency.

Moreover, because the compiler knows what you're actually trying to do, it can often perform optimizations that are not possible in C. For the example of virtual function calls, the equivalent in C (both in terms of functionality and efficiency) is calls using function pointers. The difference is that in C++ the compiler often knows the dynamic type of an object (if it's an actual object and not a pointer or reference) and can optimize away the virtual function call and replace it with a static call (or even inline the function). The C compiler is unable to do that.

So yes, there are features in C++ that have a performance penalty, but they have no equivalent in C, so the comparison is invalid.

As for ocaml or other FP languages, I think it's a good idea to try them. Besides the productivity and maintainability gains, you may also have actual efficiency benefits. Again, because the compiler knows what you're trying to do in a high(er) level language, sometimes it can perform obscure but very effective optimizations that can beat what an average or even good C programmer can do.

more than just a language performance question...

[CaptainAbstraction]

This is more than just a language question. It looks like you're starting to get the standard responses already for Java, C++, etc.

But all of these opinions presume that you're fairly experienced in these languages. Ignore them.

Language experience/familiarity is THE factor here, so don't discount it. Someone who has been eating and breathing Java would likely produce speedier code than someone who is just learning C, for example.

Your employer/client wants SPEED. This project involves hairy and complicated bit fiddling. I would suggest NOT using this project to learn a new language, for the risks outweigh the rewards in this situation.

If you choose to use a new langauge for this critical job, you're setting yourself up for disappoint. Do not forget that you're going to have to go through the all the growing pains associated with a new langauge. You're going to spend weekends tracking down (and learning from) all the newbie mistakes one makes with a new langauge. You are going to encounter new and unfamiliar bugs at all levels - logical design, physical design, semantic, syntactic.

Do you really want to spend your nights and weekends figuring out what the heck is throwing some particular JAVA exception seamingly at random? Why your C++ function template specialization is being ignored?

Learning a new language is exhilarating, but that will quickly turn to FRUSTRATION when you run into that weekend-long show-stopper bug.

With your product being measured by performance, and with deadlines looming... When it comes down to crunch-time, I think the choice is OBVIOUS!!

Choose a different, fun project to learn a new language. But for this product you're delivering, I would encourage you to stick with the tools you know and love.

Best,
Captain Abstraction

Use C

[mccalli]

Looking for an IO library standard across platforms?

#include <stdio.h>

Says it all really.

Cheers,
Ian

wow

[sinserve]

Your "speed" priority, and the binary processing bit, got me almost sold, and then
I saw O'Caml!!

You quiche eating wanker, how COULD you forget assembly? Isn't that what programming is
all about? And WHY are you comparing C to O'Caml, a fine assembly macro language, to
shity ML dialect used by equally hard-wanking mathematicians and abstractly thinking
creatures? If these wankmaticians knew how the world operated, they would not
have invented recursion let alone APPROVED of inductions as a sane, corner stone
princible in their so called "art". Induction is only possible as long as the
the "counter" register can hold your index, and recurssion is the crackwhore narcessistic
twin sister of iteration (there is nothing she does, iteration can't do with
a well placed label and a jump.)

Listen to me son, read Quine, Boole and DeMorgan, get the manual to your processor,
and "script" at the level of the ONE TRUE ABSTRACTION LAYER.

Using C++ properly????

[PD]

How can you use it improperly? C++ is an object capable language, not a strict object oriented language. If you want to use objects, then fine. If not, then please don't.

Object oriented development is a tremendous thing, useful for many things, and a marvel of overcoming complexity through abstraction.

BUT, OOP is not the solution for everything. There are many problems that don't need an object structure, and should be written another way. Above all, drop the notion that C++ should be used only a certain way to be proper. The latest cool feature of C++, the Standard Template Library, isn't even object oriented - it's GENERIC, because that type of programming just was the right thing to do for that library.

One Word:

[brunes69]

QBasic.

What speed hit?

[SIGFPE]

Last time I checked people were writing faster readable code in C++ than in C.

A smart C++ programmer can use template metaprogramming in a library like Blitz++ to automatically build code optimised for the job. To write the equivalent code in C is possible but it's much more laborious and harder to maintain.

There are good reasons not to use C++. Performance isn't one of them.