Web Interfaces for C++ Introspection?

Milo_Mindbender asks: "For a C++ application I'm working on I want to be able to pop up an interface to a class that will display all the 'tunable' parameters of the class and let me inspect/modify them, while the program is running. The catch is that I'm running on a minimal embedded OS with Open GL but no GUI library. Rather than porting a widget set or writing my own, I was thinking about having the application talk to a web browser, and then use the browser to display the GUI, take user input, and finally push the data back to the app. The classes have metadata that describes the public data locations/types so they can be accessed, but not being a web-wizard I'm not sure of the best way of generating the information I need to create the UI. My first thought is to generate HTML and push that to the browser, but it seems like there must be a better way than this, maybe someone has written a library specifically for doing this sort of thing? Any help/suggestions would be appreciated!"

Comments

Why GUI

[the eric conspiracy]

Any particular reason that it has to be a GUI? Something like an ncurses interface might be a lit easier.

Re:Why GUI

[billcopc]

I second this opinion. Remote debugging or "tuning" could be rather easily done from a serial terminal. There's no need for an HTTP protocol and the sloppy hassle of HTML forms; just a quick text console, not unlike those found in games such as Quake.

get myvar, set myvar xxyy, toggle myvar, listvars etc etc

Yeah, web interfaces are nice for end-users, but since you just want to debug your own one-off app and it's quite likely you will be the only person doing it, might as well be lazy - I mean efficient - and code the simplest interface possible, as long as it does what you need.

Re:Why GUI

[Anonymous Coward]

web interfaces are nice for end-users

No.

Re:Why GUI

[Grab]

Too right - follow KISS and you won't go far wrong.

And if you want a GUI, then it's a simple bit of Python to add a front end to that serial comms and text. A web interface is usually a pretty dumb move, unless you're sure that the person using the interface won't have the relevant software.

Grab.

This question is odd....

[Manip]

I find this question somewhat odd... The guy clearly knows C++ and is able to develop on an embedded OS which isn't an easy thing to do but yet can't write himself a simple web-server, which a lot of us learnt during our first few years of programming in something nooby like VBA or Java...

So either he isn't being completely accurate in so much that he might not be as knowledgeable as he claims or just by freak chance managed never to write a web-server in a couple of hours like everyone else...

Re:This question is odd....

[FooAtWFU]

Well, I'm a junior in college and a computer science major, studying things across the spectrum from kernels to compilers to recursive descent parsers to Turing machines and much of the stuff in between. I've had several jobs dealing with web or Intranet applications and embedded development. And I've never written a web server. I've never even considered it. I've written some pretty low-level CGI bits for C/C++ applications (parsing query strings and POSTs) and I've written some TCP/IP network communication bits, and I'm sure I could write a web server if I had to but... why should I ever need to? I've got Apache, I can look at other web servers like Tux, even IIS... and I have well-defined interfaces to these like CGI, and mod_perl, and things like JSPs for Resin. Write a web server "like everybody else?" I sincerely doubt that even one in fifty programmers have written them- and I'll even let you choose the definition of "programmers". Of all the nifty applications you could possibly imagine to write, why would you choose a web server?

Re:This question is odd....

[Anonymous Coward]

The guy clearly knows C++ and is able to develop on an embedded OS which isn't an easy thing to do but yet can't write himself a simple web-server

Sorry, you've just given yourself away as an amateur. Every professional with two brain cells to rub together quickly realises that writing everything yourself is a non-starter, and that decent libraries can make you more productive than dozens of programmers who choose to write everything themselves.

Re:This question is odd....

[Rezonant]

I'm studying for a CS masters and a UNIX-based web server is the final project of our "Systems programming in C" course. It's an optional course though. Oh and BTW, fork() and the unix process model sucks as a parallellism primitive. I love Win32 threads and io completion ports :/

Re:This question is odd....

[MarkRose]

Unix has threads, too. Recent Linux kernels have particularly fast threading.

Re:This question is odd....

[Short Circuit]

Oh and BTW, fork() and the unix process model sucks as a parallellism primitive.

You're right...you should use pthreads or nptl instead.

Some advice from someone who makes the same mistake every now and then...When you learn something in class, don't talk about it on Slashdot for at least a month. That'll give you a chance to think before you type.

In the mean time, dissing UNIX around here is likely to get you modded "Troll."

Re:This question is odd....

[Omnifarious]

Personally, I think threads are a stupid thing to use unless there's no other way to achieve your goal. I would suggest looking at the 'epoll' call on Linux and the kernel queue (kqueue?) on BSD. select and poll aren't that great.

Re:This question is odd....

[Omnifarious]

Well, I didn't assume he was asking how to write an http server. I assumed he was looking for introspection facilities like Java's, which C++ doesn't have. I wish it did. typeinfo could be that if any compiler manufacturer implemented a rich enough set of extensions to it.

It would be nice to get, at runtime, a list of pointers to functions a class supported as well as their names and type signatures. It would also be nice to be able to get a list of members and their types and their offsets from the beginning of the class.

Re:This question is odd....

[BigZaphod]

Out of the 20 or so programmers I've known rather well over my life so far, about half have resorting to building their own web server (even in languages like Perl). They've done it for kicks (because, seriously, HTTP 1.0 is *easy*) or for practical reasons (like for the sort of problem you're having). Servers like Apache and IIS are for heavy loads and high reliability - but HTTP itself is a really simple protocol. Writing your own server should not only be reasonably trivial for someone such as yourself (from the sounds of it), but also an eye opener into a whole new way of approaching your problems. You said yourself that you never even considered it. Now consider asking yourself why not?

Re:This question is odd....

[GileadGreene]

BTW, threads of any sort suck as a parallelism primitive. A better choice would be CSP-style process networks, as implemented in occam and C++CSP. Erlang implements a similar (but not identical) concurrency model.

Why would you want to use a CSP-style concurrency model instead of threads? To quote from the occam compiler homepage:

CSP has a compositional and denotational semantics, which means that it allows modular and incremental development (refinement) even for concurrent components. In turn, this means that we get no surprises when we run processes in parallel (since their points of interaction have to be explicitly handled by all parties to these interactions). This is simply not the case for standard threads-and-locks concurrency, which have no formal denotational semantics and by which we get surprised all the time.

In layman's terms, you get concurrency that can be built up from easily understood pieces (instead of a monolithically concurrent system with locks scattered throughout the code), and an underlying theory of concurrency that lets you understand and analyze your design and ensure that it is, for example, free of deadlocks (I've personally created complex networks of 1000+ interacting processes in a dynamically evolving topology, with nary a hitch). And did I mentioned that the context-switching performance of most of these systems is amazingly good?

Re:This question is odd....

[some guy I know]

The guy clearly knows C++ and is able to develop on an embedded OS which isn't an easy thing to do but yet can't write himself a simple web-server, which a lot of us learnt during our first few years of programming in something nooby like VBA or Java

I've been using C for over 25 years, and C++ for nearly 20, and I don't remember writing a web server when I was a newbie.
(Of course, the web did not yet exist when I was a newbie, so that's probably why.)
The point is, I've also written device drivers and other O.S. components, both for embedded and non-embedded systems, but have yet to write a web server (although I've written a client or two using high-level Python libraries), so for someone who went to college pre-WWW (or even pre-Internet), such a thing may not be so strange.

Also, my guess (based on my experience) is that it is much more difficult to write a web server in C (and in C++ without the STL, which may not be available for his "minimal embedded OS") than it is in the "nooby" languages that you mentioned (i.e., may take more than "a couple of hours").

Re:This question is odd....

[Anonymous Coward]

To the guys defense, I know C, C++, asm, 386 systems programming, Forth, Lua, VB, a few embedded systems, device drivers, i've written from scratch my own multithreading kernels in os projects, etc, but I would not know where to begin writing a web server either.

In the wide world of computer programming, network programming is it's own continent, and not everybody has visited.

-- another 'freak' that wouldn't know how to write a web server

Re:This question is odd....

[Anonymous Coward]

Just wait until you discover select() or poll(). You'll wonder why you ever bothered with threads and locks and all those headaches at all. Oh, and just imagine you had a language with continuations or coroutines, you'll wonder why threads where ever invented.

(Apache however does suck, because it fork()s. That means you have to explicitly put every bit of persistent data into some shared storage and lock it. A very stupid design, and guess what, select() would have simplified it...)

Re:This question is odd....

[petermgreen]

select and poll are fine for moderate load its only when you get into thousands of simultainous connections that they start to become an issue.

Re:This question is odd....

[petermgreen]

agreed also if you say wan't a web based admin interface to a daemon app then things like cgi aren't really the kind of interface you want, you are going to have to write the cgis as seperate apps then connect them to your main daemon somehow.

whereas if you code the web server yourself you can just build it into your daemon and access the daemons transiant data through direct procedure/method calls or even direct reading of global variables or fields.

and provided you don't care about things like chunked encoding or keep alive http/1.1 is pretty simple to make a basic implmentation of too (though the spec is horrible)

Re:This question is odd....

[Omnifarious]

That's true. Though I consider the interface itself to be a little ugly. :-) And using select, poll, or epoll still makes signal handlers a pain, though they aren't with kqueues.

Re:This question is odd....

[Rezonant]

Sure, but that's not traditional UNIX and is not guaranteed to exist under all unix systems. On Windows you can count on a reasonably decent threading model being there. Although it may be argued that limiting yourself to Windows is just as limiting as limiting oneself to a few particular unixes.

Re:This question is odd....

[Short Circuit]

Would you argue that Windows XP SP2 qualifies as Traditional Windows? It exemplefies thatAPIs available under one version may not be available, or even behave the same, under others.

Other examples:

• NT kernel vs DOS kernel. The Windows 95 era should have been the Windows NT4 era.
• Early versions of DirectX compared with later ones. You can't run many old games under later versions of Microsoft's "Gaming API".
• Win16 not supported in WoW for 64-bit Windows. In the Real World, there are still a lot of 16-bit Windows applications doing things like tracking inventory and controlling manufacturing equipment. The only good thing about these peoples' situation is nobody writes viruses for their OS any more.

Yes, there are valid reasons for this incompatibilities. But a developer for Windows should expect a platform that's stable for the short term, but has earthquakes every now and then.

Linux, an admittedly large subset of the UNIX world, is by comparison seismically dead.

Re:This question is odd....

[penguin-collective]

Oh and BTW, fork() and the unix process model sucks as a parallellism primitive. I love Win32 threads and io completion ports :/

I see: you indeed are a future Windows programmer, and you are ready to produce the same bloated and flaky software that your predecessors produce.

(FWIW, you don't have to use fork for parallelism on UNIX; UNIX systems have always had better threads than Windows, it's just that UNIX programmers have the good sense not to use threads if they can help it.)

Well, you don't necessarily need a web browser...

[FooAtWFU]

Instead of embedding a mini HTTP server into your application so it can talk to an application, you could just write a GUI in your favorite language (Java? C++? Java?) and talk to it with good old-fashioned TCP/IP (or even UDP) network sockets. Probably a lot simpler to define your own stuff than to deal with a big old standard like HTTP and parsing all the browsers which may be mangling your requests.

If it must be accessible via the web/intranet/browser/etc, set up some of your favorite CGI/PHP/JSP/Python/Ruby and have it talk to the application via TCP/IP and to the browser with HTTP.

Tell us about your metadata!

[Pyromage]

Well, C++ doesn't have any metadata on its own, so you must be using something third-party to create it. Since any library that provides the functionality you need will read that data, we need to know what it is!

Of course, I don't know of any such libraries, but anyway. I am very interested in how you'd be doing reflection-type stuff in C++. I'd like to investigate doing that to do mock objects, like nmock does.

Remote XUL

[dorkygeek]

You may be interested in XUL then. Especially, in remote XUL.

Also, see Remote XUL Application Development with_Mozilla I and Remote XUL Application Development with Mozilla II.

You may even be able to create the UI XML files automatically from your interfaces, using a script, or introspection.

You can then send the data back to your host, using RPCs or a REST-like interface.

Light minimal XML interface

[Zarf]

You might consider a very minimal XML interface on top of a very very very simple HTTP server. Your UI could live on a completely different device on a completely different server... or on the same server in a different application stack. You could run your service on port 8080 or something if you didn't want to run it on 80 to prevent

You could display your state as a single XML document that is gotten at a single URL. Each setting could be manipulated by posting to a URL specified in the XML document. These sets of documents and post URLs could become a simple CGI set of interfaces you could describe for anything from webservers, application servers, or other web-aware C++ programs to hook into.

Re:Light minimal XML interface

[pnatural]

This is what I was thinking, too.

I suspect the biggest bang for the buck would be embedding a python interpreter with a subset of its standard library. Figure maybe 2-3 MB VM size and about the same on disk. Everything the poster needs is in there; the web server, the xml, and the ability to update his C++ code in-process.

Re:Light minimal XML interface

[GileadGreene]

Figure maybe 2-3 MB VM size...

Better hope that "minimal embedded OS" he's using isn't running on a "minimal embedded system" as well. Otherwise 2-3 MB of VM may well be out of the question.

Re:Light minimal XML interface

[Zarf]

Can he open a socket? Can he put characters to it? Can he read characters from it? Maybe 2 or 3 MB is too little to do that in. What do you need to read and write text? Maybe 128 MB or so?

Re:Light minimal XML interface

[Z34107]

Can't tell if you're joking or not, but (modern) computers since the beginning of time read and wrote text. The mainframe at MIT worked with text (and much more) back when the first megabyte stick of memory cost over a million dollars, without adjusting that sum for inflation. Of course, you need enough memory to hold the text you're working with, but to work with it, you need, at most, kilobytes.

Re:Light minimal XML interface

[Zarf]

So it sounds like most devices could read and write text now-a-days right? If that's so then maybe they could write text that was generically formatted into some kind of mark-up language... an ML. We want something really generic so let's call it "XML" and all this XML would do is represent a data structure. You would print it to a socket. So even if you had a very teeny tiny embedded device that at least could do networking, could at least use a couple of Kbytes of RAM that device's data structures could be read by another program. Perhaps a bigger heavier program living on a box with lots more space and more readily modified so the output could be "pretty printed" to a data browsing program of some kind.

Getting data back into the teeny weeny device could also be accomplished by reading data posted to the socket. So the idea here is that if you have an itsy bitsy teeny weeny device that is beefy enough to read and write to a socket on a networking infrastructure of some kind... then you could use a different system to produce UI things like web-pages.

The crazy idea here is: why not use XMLHTTP everywhere? DO NOT spend your time trying to make pretty webpages on an itsy bitsy device that will have your sloppy gawd-awful HTML burned into its firmware... instead make generic web-service interfaces that a real (tm) web person can hook into using some kind of XML service architecture.

Re:Well, you don't necessarily need a web browser.

[Anonymous Coward]

HTTP is hardly a very big standard. A few header fields (which he could even ignore for his application), and that's it.

GLUI

[tfinniga]

Have you looked at GLUI?

Re:GLUI

[destuxor]

I've never heard of GLUI and never run into a program that used it; a more established library is GtkGLExt, and the most common GTK GL interface is gtkglarea, which has become a part of Gnome.

Re:GLUI

[MS_is_the_best]

Although GLUI is an option for solving the problem it is not an answer to the question.

Just link something like libhttpd. You can find enough information in the documentation to get you started. Just use httpdPrintd(server, "Mijn var is %s, foo) to show your variables and myvar = httpdGetVariableByName(server) to get posted (changed) values back.

Re:Well, you don't necessarily need a web browser.

[plover]

To solve his problem he wouldn't need to "parse all browser's requests." He could code it simply for the one he uses. Obviously if this is truly a minimal system, a pre-packaged web server is likely to be more than his resources can bear. But using a web interface eliminates the need to write a client side app.

So yeah, write a text interface to a socket, and telnet to it from off the box. It may not be clickably pretty, but he can probably solve his problems quickly. It's what I used to do twelve years ago when I had a similar situation, and it worked just fine then.

It would help....

[aminorex]

It would help if you said what the OS was, or at least specified what APIs were available natively. I would suggest that you should not discount the option of using X11 instead of a browser, at least not without evaluating the pros and cons.

Assuming only a socket API, the easiest thing to do is use a simple text-based protocol, talking to a minimalistic TCP server, probably written in PHP, Perl, or Java, and acting as a FastCGI script. You'll find it much quicker and less error prone, chatty, and code-heavy than any markup-based protocol. It's so trivial that it's not rational to even consider any sort of higher-level helper API, I think.

Re:It would help....

[Brandybuck]

It would help if you said what the OS was

He did, it's a minimal embedded OS. There are a thousand and one of these out on the market, plus tens of thousands of others written from scratch in-house.

What about ICE or CORBA?

[motus]

I think the challenge here is not how to represent the data visually (i.e. web/GUI), but how to control a C++ object from the remote application. For that puropse, I would suggest CORBA. You can define all controllable classes in CORBA IDL, compile it into C++ code and integrate with your existing application with minimum efforts. CORBA client should not necessarily written in C++ - it can be Java or Python, for example. I have very good experience with omniORB (http://omniorb.sourceforge.net/). It supports both C++ and Python, and I use a bunch of Python scripts as a test harness for my C++ CORBA services. Besides omniORB, there are lots other decent implementations of CORBA in many programming languages (http://www.omg.org/technology/corba/corbadownload s.htm).

PS. Good alternative to CORBA is ICE (http://www.zeroc.com/ice.html), which is basically the same thing as CORBA, and founded by one of the CORBA gurus. ICE has much better C++ mapping, and lots of other nice features.

Hope this helps!

use openGL

[Lehk228]

if you are doing openGL stuff how about throwing in an interactive debug mode. I am pretty sure openGL can sned lines of text in and out just have the app load a translucent console window when launched with --debug

Perl and GUIs

[kwench]

I had a similiar problem with how to write a nice GUI in Perl and have an application that can run over a network? I used a webserver and CGI.pm. I don't know whether there are CGI.pm-like libraries for C++, though...

Database

[Profound]

Instead of:

Webapp Your App

Consider:

Webapp Database Your App

Now you have 2 simpler problems: get your app to get its data from a database, and get your webapp to display and modify values in the database.

Re:Database

[Anonymous Coward]

That's not going to work. Then he incurs a database hit every time he wants to use one of those changeable parameters.

Re:Database

[Profound]

He didn't mention any performance limits in his post.

And by database I don't mean say, ASE, MSSQL, Oracle or DB2. You can get very small, very fast in-memory databases.

Re:Database

[Anonymous Coward]

tell that to all the fucking PHP/MySQL weenies out there, who think connection pooling is when you drive each other to work....

Re:Well, you don't necessarily need a web browser.

[Anonymous Coward]

If he still wants to do the web interface, I'd probably still go the basic TCP/UDP/whatever route, and then have a seperate small web server that that translates between the simple interface and HTTP.

That way you can keep the HTTP/UI code seperate and isolated. Less hair in your core code is a good thing.

Use Qt

[e8johan]

Use Qt and declare your tunable parameters as properties. Writing a minimal server using Qt is trivial.

How we do this

[Anonymous Coward]

At a place I contract, we do exactly the sort of thing that you want to do. The steps we take are:

1. Define a base class that describes a schema that can be used to store arbitrary data (effectively keeping location and run time type data well defined)
2. All users classes are defined off this base class. *Data* that is to be made externally visible is created in terms of the data schema
3. *Classes* that are to be made visible are instantiated via a class factory, and pointers to them kept in a hash.
4. A separate thread receives requests to inspect/modify data within a specified class. This thread looks up the class in the hash, and then processes the request. But only data in the schema can be accessed. By requesting the schema details from the object, the browser doesn't have to know ahead of time what it is trying to look at.
5. In our case the comms link is via a propriatary protocol, but the data is sent in XML. (Personally I think that this sucks as we can generate megabytes of data when logging, so the transport efficiency is way down. )

I am sure for the right price we can sell you our secrets. But on the other hand, I have seen the inside of all of this mess and it is *not* pretty.

Have you looked at thttpd?

[swillden]

It's a tiny web server with CGI support. You didn't say anything about your target platform, so it's possible that it's still to large, but it's only about 70KB compiled. It would have to be ported to your OS, of course, but at only 8000 lines of code, that shouldn't be too difficult.

As for what to serve up, in the interest of keeping the code small I would suggest defining a custom XML markup rather than HTML. That way your on-board code doesn't have to generate all of the formatting markup to make it look pretty. Instead, for simple presentations you can probably just use a CSS stylesheet (linked from the device-served XML, but not necessarily provided by the device). For more complex presentation, you can use an XSLT stylesheet (again, linked from the), plus CSS for formatting. For the latter, you'll need an XSLT-capable browser, but Firefox and IE6 both fit the requirements, so that's fairly easy.

interactive scripting

[ackdesha]

I'm not sure if I'm on the same page, but could you create bindings using ruby or python and modify your class attributes using an interactive shell like irb?

Uhh, AJAX?

[mosel-saar-ruwer]

Rather than porting a widget set or writing my own, I was thinking about having the application talk to a web browser, and then use the browser to display the GUI, take user input, and finally push the data back to the app.

WARNING: BUZZWORD COMPLIANCE ALERT!!!

Uhh, isn't this exactly what AJAX was invented for? If his "minimal embedded OS with Open GL but no GUI library" had an eency-weency [itsy-bitsy?] little webserver, like thttpd, with some sort of an AJAX extension, then he could interactively push [& pull] all of his shiznat to a web browser that lived on a full-featured client operating system.

PS: If youse guys haven't played with it yet, the W3C DOM is the bomb when it comes to this sort of thang. You can do some stuff with the "childNodes" thingamabob that will make your jaw drop.

PPS: Isn't it a little self-contradictory to utter the phrases "minimal embedded" and "Open GL" in the same breath? Or am I missing something?

PPPS: OpenGL [or even VRML] over AJAX via thttpd - now that would be helluva project for a 40-year-old browncoat living in his Mom's basement...

Re:Uhh, AJAX?

[Anonymous Coward]

You are a fucking monkey.

Please do not, ever, post again. Anything.

Re:Uhh, AJAX?

[Anonymous Coward]

Agreed

RPC Server

[daaku]

Use something like libhttpd or thttp to embed a webserver. Using ideas based on REST, design a RPC system (can use XML-RPC or if you're bent on making it a web based frontend, then maybe JSON-RPC) to communicate with the GUI.

Slight variation on the 'generate HTML' theme

[ArwynH]

How about minimal server-side XML + Client side XSLT?

Requirements:

1. minimal http server (There is bound to be BSD licensed code for this somewhere on the net)
2. Code to generate simple (possibly flat?) XML file. You don't need a library for this, you could just pretend it's a text file if the XML is simple enough.
3. XSLT and CSS stylesheets

Now all you have to do is serve the XML and stylesheet, attach the stylesheet to the XML file and watch any XSLT-aware browser do the heavy work of transforming that simple XML into beautiful XHTML (or HTML) for you.

Benefits:

1. server side code is farely minimal (just a mini-http server and xml generation code) and some of that (mini-http server) can probably be re-used from elsewhere.
2. Full content/UI seperation. Client doesn't like the UI? just change the XSLT/CSS till he does, no need to re-write code. You could also write small client program that uses the XML data if client doesn't like using a browser, again no need to re-write code.
3. Less time-consuming than writing a full-scale UI

Problems:

1. Requires XSLT-aware browser. Mozilla and Opera support it no probs. IE supports it aswell, but has a few glitches (suprise!).

Why re-invent the wheel writing C++ UIs and stuff when there is plenty of simpler ways out there.

E-Z way

[Anonymous Coward]

Assuming you don't have a lot of people looking at the same thing at the same time (e.g. just yourself looking at your objects while developing), the easy way is just to have a simple web app save data in a file and your C++ program to read it. Just flock() the file during either read or write.

The web server script can just save it like this and then you sscanf:
0x1234 // pointer addr
34 // int value, etc.

Or by whatever other method you use to serialize and deserialize.

Or, you can have your C++ code do a straight memory dump of an object if your web script can parse it with the padding. You just have to make sure you dump each object referenced -- so if name is a char *, and you want to change it, you change the chars at that addr, not change the addr of name, assuming you have enough space. Using a C++ std string that dynamically resizes complicates things.

C++ Introspection

[Anonymous Coward]

See the SEAL Reflex project, the OpenC++ project, the PUMA project(now in AspectC++), Arne Adams' reflection library, and XVF by Kurt Stephens. These all work and provide introspection to some degree. There are other projects like Stroustrup and dos Reis's The Pivot and Vandevoorde's Metacode that may make it into future C++ standards to make it easier to provide good introspection support.

SEAL Reflex http://seal-reflex.web.cern.ch/seal-reflex/

OpenC++ http://opencxx.sourceforge.net/

AspectC++ http://www.aspectc.org/

Reflection library http://www.arneadams.com/index.html

XVF http://kurtstephens.com/research/paper/xvf_paper/

The Pivot talks http://charm.cs.uiuc.edu/patHPC/slides/stroustrup- a.pdf
http://www-unix.mcs.anl.gov/workshops/DSLOpt/Talks /DosReis.pdf

Metacode talk http://www.open-std.org/JTC1/SC22/WG21/docs/papers /2003/n1471.pdf

I guess I wasn't clear enough

[Milo_Mindbender]

Sorry, I guess I missed a few details about what I was looking for. The main thing I was interested in was some library that would autogenerate the http/xml code necessary to display a variable inspection/modification GUI in a web browser. It would be nice if it was already integrated in a library that also did the minimal web-browser stuff needed to send the page to the browser, fill in the data and then manage getting the data back.

Specifically, each object in the app has metadata describing the class members it wanted to appear on this interface, their names, types, pointers...etc. All this info is used internally by the app for other things like delivering network messages, state saving and such. I'd like a library I could hand this metadata to which would then algorithmicly layout a simple GUI, send it to a browser, possiblly update the data if it changes on the app side, then accept changed values back from the user. The functionality is somewhat similar to a remote debugger, but without execution control and with the target app controlling what the user can see and tweek.

Can't really use curses or X directly because the embedded platform doesn't have them. It has an open GL driver/screen and I want to reserve that for actual app output so the UI for displaying/tweeking data needs to be on some other machine. Yes, I could whip-up a windows/linux network app that would talk to the embedded platform and display the data using a native widget set, but using an (already written) web browser seemed easier.

Forgive me if I'm describing something that sounds stupidly easy to do, I just haven't spent a ton of time coding up web-apps (I mostly do embedded stuff) so I'm not all that familiar with all the web standards and tools for doing forms & such. I could write the minimal http server, it's generating the stuff to serve up that I was more sketchy about.

Re:I guess I wasn't clear enough

[petermgreen]

X11 is network transparent so as long as you can build the lib on your target you should be able to use it with the X server on another machine

Re:I guess I wasn't clear enough

[God of Lemmings]

The web server stuff is fairly simple, you just make a server program sit and listen for an http request interpret it and spit back the html at the client. Plus if you need to, you can thread/fork program to deal with multiple requests.

As for how and what to do, look at this:

http://72.14.203.104/search?q=cache:sZENqumprsYJ:w ww.cs.rpi.edu/courses/sysprog/sockets/sock.html

and this:

http://72.14.203.104/search?q=cache:FztWl-yhet8J:w ww.cs.wustl.edu/~schmidt/cs544/socket1.html

After that you just write a few functions to produce the output you want, close the socket and wait for the next request.

Re:I guess I wasn't clear enough

[BP9]

In a current embedded product that is mostly C++ based I'm working on I had the same issue. I wrote a command line interpreter that talks to a telnet client; it works fine, but it was getting painful writing ascii 'reports' for complex classes that were readable. Writing the output via html and letting the browser deal with layout (using nested tables for instance) is close to trivial and makes formatting almost a non task (a little CSS can even dress up an otherwise plain pages).

As to how to do the introspection, there is a library called BFD (part of gnu binutils) that understands how to plug through the debug symbol tables, and in a previous product we did use it for this sort of thing, but IMO its way too painful and not really intended for this sort of use. It ended up being a giant rathole, although we got what we wanted in the end (ability to do a formatted system dump on a saved raw system image using compiler metadata to find the interesting stuff and print it out nicely, think the old unix 'crash' command).

The current product has a few hundred classes that are worth looking at in a running system, I've used the convention that operator<<(std::ostream&, class) will serialize out the class ("programmer visible introspection?") in an easily parsable format, specifically TCL style lists of key/value pairs for each (interesting) field in the object. For aggregated objects (or optionally referenced objects) you output the name of the object in parent class then call operator<< on the child object (... elephants all the way down). Its relatively easy to write these serializers, example:

std::ostream&
SomeClass::put(std::ostream& s) const
{
s << "this " << this
<< " field1 " << field1_
<< " aggField2 {" << aggField2_
<< "} field3 {" << field3_
<< "} field4 {" << field4_
<< "} colids {";
util::printcollSpec<std::vector<jint>, util::PrintWsList, util::Direct>(s, *colids_);
... etc ...
return s;
}

Boost also has a serialization library that may automate some of this (its fairly new and I haven't looked into it yet).

If you have threads available (we do) and don't mind linking with the TCL library the tclhttpd server is a fairly comprehensive embedded web server that can run in its own thread (using blocking I/O and not getting in the way of any system wide epoll/event loop). TCL scripts are triggered by various urls (it also has an asp/jsp like thing which hasn't been useful in our environment), these TCL scripts suck the serialized version of the object to look at into a TCL associate array ("array set t [xyzStatus]") and then a small hunk of code generates the HTML from stuff in the array. For most classes I just use a short generic routine that makes a table from a TCL array (couple of lines), and understands about sublists, creating a sub-table when it detects an aggregated object. Putting all the generation of HTML in TCL script makes it pretty easy to whip up a page in short order.

There's a fair TCL library that deals with some of the arcana of form generation (which isn't that hard), package html; it has thin but barely adequate docs (if you don't mind reading the code sometimes). Posts are easy to deal with to change debug settings. You can push javascript that talks to the DOM out to the browser to get more dynamic and interactive displays ("AJAX").

On a prior project I had a go at doing this client/server as some have suggested, the overhead of writing RPC for the various classes was somewhat higher than the simple serialization stuff above. The main issue was that changes to displays require a redeployment of the client app on several platforms. While some random developer would be happy to stick some support code in his class and maybe write some TCL scripting to make a web page to talk to it, almost no one was willing to

Suggestions, request for more details

[insulanus_hailstorm]

This is a great question. Sounds like you have a few different problems:
- Changing the runtime state (introspection)
- The GUI

First, I'd like to ask how much memory you have available on the embedded system, and how many classes you are planning on using this introspection approach on. Do you have threads or processes on the embedded platform? Does your parameter tuning program have to run on multiple platforms?

If you have enough memory, you may want to embed something like lua, tcl, ruby, guile or python and wrap them in C++, then introspect those classes. I know this sounds like overkill, but it will seperate the dynamically changing classes from the rest of the program. As I'm sure you know, it is a pain to manipulate runtime state in C++. Pain will be eased by using SWIG

Do inherit from a class that has something like read_state() and write_state() in some format you can understand. Check boost - they may have something usable for you.

Also, think about whether it is strictly necessary to modify the various attributes on the fly with custom code. What I mean is, if you have a reload()/init() (better yet, the constructors themselves) method in your classes, you can have your GUI write out what amounts to a config file, and use the same interface to change parameters that you do to initialize the objects. This will help you keep your class invarients better.

Whatever you do, try not to let any of the UI muck get tangled in your embedded app. (For example, having your embedded app spit out html directly). Even if you have to write a seperate program to run on the embedded platform, that would be better than having your app communicate with your control program (be it a browser or native UI) directly. ( Unless your embedded app is already a web server :P )

It will almost certainly help you to use a command-line program (write it in some scripting language to save your time) as the first incarnation of the properties twiddler. Use a simple ascii protocol. This will be easy to convert to html later on. You will be happy to have your command line tool while you are debugging the browser interface.

If you have the ability to run a seperate program that just proxies your simple protocol to the web browser, so much the better. Thttpd is your friend here. It is one of the easiest HTTP servers programs to port to a limited platform.

eg. \n
class: foo
attrib: blarg=jack
attrib: sam=600
\n

A web interface has the following advantages:
- Easy to update/change the code
- Cross platform

If you haven't done any client UI stuff on windows or Linux, believe me, now is not the time to learn MFC or X11 or Qt or GTK+ etc etc. The only app I'd attempt with "native" widgets in that case would be with Tk.

Good luck!

P.S. The efficiency hits of all the above will almost certainly be worth it.

Markup Languages

[Z34107]

XML is pretty and human readable, but there's a lot of space wasted to make it so - especially for just representing a small structure. A binary markup language would probably be better. For example, two bytes of ASCII would get you the < and one character of an XML tag (even less in Unicode) whereas with a special binary format, you could represent the tag as 1 "start of tag" byte followed by 1 "type of tag" byte. Not pretty and not human readable, but you're proposing having another machine parse it anyway, right? It could save one metric heckuvalotta RAM.

Plus, networking drivers, even minimal ones (be they TCP/IP, NetBIOS, NetBEUI, whatever) plus the drivers to interface with the NIC hardware can be fairly large. If the RAM is limited, and therefore what you're transmitting is small, it would probably be easier just to tether it to the "beefy machine" with a serial cable.

Not pretty, but if space and processing power is truly at such a premium...

Re:Markup Languages

[Zarf]

Not pretty, but if space and processing power is truly at such a premium...

At this point I'll point out that the original poster was talking about deploying an entire HTTP server with HTML in the device. If it was even possible to do that, then we know the device was in a class that would allow such waste as an XML export. His idea of pushing HTML from his device to the browser is perhaps one of the single worst ideas I've ever heard. If this is a one off project that nobody cares about, then fine, but if anyone is going to have to live with this product... pushing HTML from an embedded device is a horrifically bad idea.

Text is beautiful in that it never suffers from endian or other platform issues. It is, however, larger and requires parsing. If you use the markup language you won't have to map the datagrams between Ada and C and Irix and Linux on the x86 in 12 years when your original embedded data source is still in use but your middle teir servers that parse the data source are utterly obsolete.

Text may cost more, but proper use of text would have saved at least one embedded system I know of. When we're talking about applications that will live decades and when we're talking about a system where an embedded HTTP server isn't outlandish... text makes sense. Text doesn't suffer from endian issues, cryptic formats, or the random undocumented datastructure.

There are reasons for each type of data representation, but, for this type of use text is king.

If we were really talking about a system with no networking on it then talking about XML would be silly, but no more silly than talking about an HTTP server on such a device. Never ever, ever never, never ever use a binary format to transmit data you're just going to plop on a web-page. Never embed an HTTP server and HTML documents onto a device if you can at all help it. If you find yourself stuffing HTML onto a device give yourself a way to do maintenance. Somebody might think your webpages are ugly.