Slashdot Mirror


Open Source Camera For Computational Photography

David Orenstein writes "Stanford Computer Science researchers are developing Frankencamera, an open source, fully programmable and finely tunable camera that will allow computational photography researchers and enthusiasts to develop and test new ideas and applications — no longer limited by the features a camera manufacturer sees fit to supply. Disclosure: The submitter is a science writer for Stanford and wrote the linked article."

16 of 167 comments (clear)

  1. hackable cams already available by v1 · · Score: 2, Informative

    My PowerShot S3-IS is scriptable. (example) And it's not even a cutting edge camera. Lots of cams support scripting.

    --
    I work for the Department of Redundancy Department.
    1. Re:hackable cams already available by mrmojo · · Score: 5, Informative

      The existence of prior art doesn't mean we shouldn't try and do it better. There's plenty of related work and similar projects.

      We're aware of chdk (and have used it for a bunch of stuff), and it's close in some respects, but it's not the same thing. Chdk doesn't turn your camera into a fully functioning linux box, which is part of what we're trying to do, though this has also been done before sans viewfinder (www.elphel.com). You can plug random stuff in over USB, you can control the sensor with extremely low latency (by hacking the kernel if all else fails), you can ssh in, you could even run a web-server off your camera if you wanted to like the elphel cameras. Last week I plugged an SSD drive in over USB (alas no sata interface yet) to save off raw data faster. It's a fairly standard linux so it just worked.

      You also have a lot more compute than you might get in something like chdk. You have access to a unified shader architecture GPU, a DSP, a CPU with an SSE-like vector coprocessor, and a fixed-function set of image processing tools (like histogram generation).

      The other half of what we're trying to do is make a really good API for a programmable camera, including stuff for synchronization of multiple external devices (eg flashes), optimized image processing routines, frame-level control of the sensor at high frame rates, and camera user interface stuff, including physical widgets like buttons and dials (we use a phidgets board for this).

      - Andrew (One of the grad students working on the frankencamera)

    2. Re:hackable cams already available by mrmojo · · Score: 2, Informative

      Well sensor noise comes in two main flavors - read noise and photon shot noise. The photon shot noise is inherent in the light - photons come in semi-randomly. The read noise is noise in the sensor circuitry.

      If you take 100 pictures with 1/100s exposures and average them, then in terms of photon shot noise it's the same as taking a 1s exposure. You've recorded the same number of photons. However, you paid the read noise penalty 100 times, and while you're still averaging that away, you start with a lot more than if you just took a 1s exposure.

      So in the end, your synthetic 1s exposure is noisier than a real 1s exposure, but much less noisy than the longest hand-held exposure you could normally take (~1/60s, depending on focal length).

      So it works pretty well.

  2. Nikon offer an SDK for many of their cameras by jamesswift · · Score: 4, Informative

    http://sdk.nikonimaging.com/apply/

    D5000, D3x, D90, D700, D40, D60, D3, D300, D200, D80
    And NEF (RAW) files

    --
    i wish i could stop
  3. Re:Do want by Entropius · · Score: 4, Informative

    The Canon lens-to-camera communication protocol has been reverse-engineered for a while. Manufacturers like Sigma and Tamron are making (very good) lenses compatible with Canon (and everyone else's) bodies.

  4. Re:Listen up camera manufacturers by mrmojo · · Score: 5, Informative

    I hear you - we want the same thing. Our target is basically a Nokia N900 (which covers 1, 2, 3, 4 and runs linux to boot), plus a much higher quality sensor and lens.

    - Andrew (one of the grad students working on the project).

  5. Re:Do want by mrmojo · · Score: 4, Informative

    It's not a Nokia imaging chip, it's just the one that happens to be used in Nokia N95s. Aptina makes it and sells it to anyone who wants one. They do make you sign an NDA to get the full data sheet, but that's pretty much impossible to avoid.

    As the poster above mentioned, Canon lenses have been thoroughly reverse-engineered.

    The lenses would be fairly easy to swap out for a different optical system - we communicate with the lens controller over a simple serial link. The sensor is more involved - for one you'd need a linux kernel driver for your new sensor. Also, it's a pain to properly mount a sensor and get the all support circuitry working. None of it is secret or proprietary though, beyond the NDA you usually need to sign to get the register map for the sensor you want to use.

    - Andrew (one of the grad students working on the project)

  6. nice PR stunt by speedtux · · Score: 2, Informative

    This sounds like an academic trying to make a name for himself again by labeling something that already exists with his own label. "Computational photography"? Well, how exactly did digital photography ever work without that?

    Open source camera OS? Nice try, but the reason manufacturers haven't standardized on anything yet is because the technology keeps changing.

    However, FWIW, Canon cameras effectively can be reprogrammed using the CHDK firmware.

    1. Re:nice PR stunt by mrmojo · · Score: 5, Informative

      Computational photography is the accepted term for this subfield of computer graphics and computer vision: http://lmgtfy.com/?q=computational+photography

      Secondly, we're not making an open source camera OS for existing hardware, we're making camera hardware that runs an existing open source OS - linux - with particular drivers and APIs to help you program the camera.

      We're very well aware of CHDK and have used it for many projects. This is not like that (I have an earlier post that elaborates above).

  7. Re:Why use the EOS mount? by Anonymous Coward · · Score: 1, Informative

    Because the best lenses in the world use the EOS mount. The EF mount allowed Canon to pioneer things like ultrasonic focusing motors and image stabilizers.

    dom

  8. Re:Do want by mrmojo · · Score: 2, Informative

    You can find such lens specifications available for most lenses in patent databases. The patents list the curvatures, indices of refraction, etc, etc. That's kind of the point of patents - when used correctly they remove the need for trade secrets. I'm not sure if this holds true for Canon lenses in particular.

    I agree with you that many aspects of the hardware are not as open as they could be. However, we're trying to make a camera that actually works well as a camera without reinventing too many wheels. We'd also like to be able to replicate and distribute the camera to other researchers. That means off-the-shelf parts that anyone can buy, like Canon lenses.

    - Andrew (one of the grad students involved)

  9. Re:Listen up camera manufacturers by RuBLed · · Score: 2, Informative

    The size and weight difference between those mediocre Powershots and LX3 is almost negligible. LX3 is still pocketable. I could say that LX3 is a superb camera vs those mediocre Powershots. Powershot G10 would be the closest competitor during their launch last year but most agree that the G10 would be harder to pocket than an LX3.

    I own an LX3 and had seen/felt a G10. I bring my LX3 with me almost everyday. Could not say the same with the DSLR which rarely see office/street time.

  10. Re:Listen up camera manufacturers by Jeppe+Salvesen · · Score: 2, Informative

    Agreed. Not everyone wants a huge camera. Not even a over-the-shoulder-sized superzoom or micro four thirds one.

    However, interestingly, the point-and-shoots waste a lot of resources and space by exceeding the diffraction limit on common apertures. Plus their "noise-reduction" algorithms is really all about selective downsampling while maintaining file size when you operate within non-limited apertures.

    You buy a 12mpx point-and-shoot, but the files themselves are closer to 6-8 mpx in terms of resolution.

    --

    Stop the brainwash

  11. Re:Similar work for Canon cameras... by Cheesetrap · · Score: 2, Informative

    You could just use a card reader, but the camera craps the pictures into 234897234 different folders -- and he's nowhere near good enough with computers to be able to deal with that.

    One would assume the camera generates unique (or at least very seldom-repeated) filenames for each photo... I don't see what is difficult about a recursive 'move' command? I'm a complete noob @ programming but it only takes a one-line batch file..

    <20 minutes later>

    Okay, damn you for making me work it out, here you go :P

    File: dumpdir.bat

    @echo off
    set listfile=c:\temp\list.txt
    if "%1"=="." (echo %cd%>%listfile%) else (echo %1>%listfile%)
    dir %1 /b/s/a:d>>%listfile%
    for /f "tokens=*" %%s in (%listfile%) do (xcopy "%%s"%4 %2 %3 /v /f /z)

    And this can be called with:

    Normal usage:dumpdir path1 path2

    Test mode (lists files only): dumpdir path1 path2 /L

    Restrict to matching string: dumpdir path1 path2 /R \*.ext (backslash required)

    Note that path1 and path2 must be enclosed in quotes if they include spaces - and if you are running the batch from the directory you are wanting to consolidate, you can use a period instead of typing out the path (e.g, dumpdir . c:\pics).

    Hope that helps someone ;)

  12. Re:Always wanted a printer by ledow · · Score: 2, Informative

    Because a printer costs almost nothing nowadays, certainly less than its component cost to a hobbyist when you can get Lexmark's etc. for about £20 brand new. Old printers are a great source of stepper motors because of this. The major problem is the ink, which is the hard bit to make effectively and cheaply on anything other than a mass scale - so actually with a £20 Lexmark printer and some "clone" ink refill, you've basically got something orders or magnitude more efficient than you could ever make yourself.

    The tech being "open source" is a different matter, but we all *know* how printers work. An inkjet is no different to a dot matrix, except instead of hitting a physical pin into an inked ribbon, the carriage heats the ink through a very fine nozzle (probably out of the hobbyists reach without considerable skill/expense). The basic technology is a geared-down belt on a stepper motor. Try getting that working to within 300ths of an inch (300dpi) for less than £20. After that, the actual imprinting technology is either out of reach or difficult to manufacture / time accurately. Then you need control circuitry (WAY more than £20 worth of chip / effort / programming). Then you need the consumable in the first place, which doesn't clog, dry up, leak, etc. And laser printers? Don't even go there for hobbyist use.

    Even back in the primarily-hobbyist days, "normal" types of printers around weren't in the hobbyist scene because you couldn't "make" them - the ZX printer on the Spectrum is one example - aluminium-coated paper which was printed on by an electrical "spark" because that's the only way it could be made affordable. Printers are one of those things that you can't make for anywhere near the price that 10 or 12 throwaway ones wouldn't cost. You don't gain anything at all. It's like saying, why doesn't someone make an "open-source" car? Because at the end of the day to make it would actually cost you more than just buying a new car anyway.

  13. Re:Listen up camera manufacturers by hughk · · Score: 2, Informative

    I like a larger sensor, but don 't need the extra pixels. It's partly down to the high pixel density why we have so many problems with low-light noise.

    --
    See my journal, I write things there