High-speed camera - First video! / Projects / Forums

Forums

4hv.org :: Forums :: Projects

« Previous topic | Next topic »

High-speed camera - First video!

previous 1 2 3 4 next

Move Thread

LAN_403

tesla500

Sun Feb 24 2008, 07:41AM

Registered Member #347 Joined: Sat Mar 25 2006, 08:26AM
Location: Vancouver, Canada
Posts: 106

Thanks guys!

Once I get the project to a more completed state I'll post all the design info.

Progress has been fast lately as I've just gotten out of school, and have been looking for a job. I recently found one, I start this Monday actually, so progress probably won't be quite as fast anymore. I started on this project about a year ago, and I slowly got most of the subsystems built, like the RAM controller, Ethernet, and the embedded CPU. So most of the stuff required was already done when I posted here about the project.

David

Bjørn

Sun Feb 24 2008, 03:58PM

Registered Member #27 Joined: Fri Feb 03 2006, 02:20AM
Location: Hyperborea
Posts: 2058

This is a very good project, well done.

There is some slight flickering in the video, I assume it is caused by the light source.

Would you consider adding some outputs that goes high in sequence, changing with every frame? My idea is to slightly modulate the colour of the light source between frames, That way it is possible to make a "colour" movie without any loss of framerate or increase in data size.

tesla500

Sun Feb 24 2008, 06:49PM

Registered Member #347 Joined: Sat Mar 25 2006, 08:26AM
Location: Vancouver, Canada
Posts: 106

Yes, the flickering was due to the lights. The halogen lamps I used must have very thin filaments, so they change temperature significantly throughout the AC cycle.

Your idea of cycling through colored lights is interesting. In fact, one standard that was in competition with NTSC proposed such a system, but with rotating color filters in front of the camera and CRT display. Reconstructing the color would be easy for static sections of the image, but would require some sort of motion vector compensation to translate pixels over to keep up with fast moving objects.

The easier way to to color is to get the color version of the image sensor I used. It has a Bayer color pattern, which almost all color image sensors use. Basically, half the pixels are green, 1/4 red and 1/4 blue, arranged in the following format:

RGRG . . .
GBGB . . .
RGRG . . .
GBGB . . .
. . . .
. . . .
. . . .

This way, the data rate and resolution are the same as with the monochrome sensor. With an interpolation algorithm, you can recover RGB values for each pixel, with very good quality. I would have gone for a color sensor, but this FPGA isn't big enough to do the interpolation real-time for the live preview, and doing it in software would be very slow.

David

Dalus

Sun Feb 24 2008, 07:01PM

Registered Member #639 Joined: Wed Apr 11 2007, 09:09PM
Location: The Netherlands, Herkenbosch
Posts: 512

Why can't you keep the preview image in greyscale and let your pc do the interpolation.

Bjørn

Sun Feb 24 2008, 07:22PM

Registered Member #27 Joined: Fri Feb 03 2006, 02:20AM
Location: Hyperborea
Posts: 2058

I like monochrome sensors. The colour filters must filter away most of the light so they will reduce sensitivity significantly both in visible and invisible wavelengths. And if I was going to use monochrome red light for some reason the resolution would be reduced significantly.

I don't agree completely that a PC is slow, in many cases it is faster than even quite large FPGAs. My CPU alone is capable of up to 40 GFLOPS, and the graphics card another 320 GFLOPS on top of that. A CPU can also use "trick" algorithms that needs far less computations than what is required with hardwired algorithms.

That is 625 floating point operations (on the CPU alone) for each pixel when running at 320x200 1000Hz. Unless the PC is really old there is hardly anything in the line of picture processing that will take any noticable time at this resolution.

tesla500

Sun Feb 24 2008, 09:03PM

Registered Member #347 Joined: Sat Mar 25 2006, 08:26AM
Location: Vancouver, Canada
Posts: 106

You're right, color filters let through less than one third of the light. Monochrome sensors are definitely nice for high sensitivity. There will always be a trade-off somewhere when you want to capture color images, the best option is to have the a choice of either a monochrome or Bayer pattern sensor, depending on what you want to do with the camera.

The PC definitely is fast, I was referring to the 50MHz embedded CPU in the camera. Getting a live preview on the camera's monitor with in-camera software processing would be rather slow.

David

Bjørn

Sun Feb 24 2008, 10:02PM

Registered Member #27 Joined: Fri Feb 03 2006, 02:20AM
Location: Hyperborea
Posts: 2058

Yes, the Microblaze at 50 MHz would be slow for that sort of thing.

One other thing, what software are you using for the FPGA, Webpack ISE?

tesla500

Mon Feb 25 2008, 06:13AM

Registered Member #347 Joined: Sat Mar 25 2006, 08:26AM
Location: Vancouver, Canada
Posts: 106

I'm using the regular version of ISE, along with Synplify for synthesis, Chipscope for hardware debugging, and XPS for Microblaze. The Xilinx synthesizer is not all that good, it's slow, and produces worse logic compared to most 3rd party synthesizers. Chipscope is invaluable for debugging the hardware.

David

mikeselectricstuff

Fri Mar 07 2008, 10:36PM

Registered Member #311 Joined: Sun Mar 12 2006, 08:28PM
Location:
Posts: 253

Nice project - as it is mostly based on a standard devboard, it would almost certainly be worthwhile for you to get some sensor PCBs made to sell - I'm sure there would be plenty of takers. The 64MB is a significant limitation though - it's really nice to have enough buffer to cope with human reaction time when something interesting happens..! Maybe look at using an alternative FPGA board with more RAM, or do a complete PCB with the RAM on as well...
Something else that may be worth thinking about - as HS material often has lots of 'nothing' outside the area of interest ( spacially or temporally speaking), I wonder of some sort of realtime compression might be viable - something simple like runlength or frame-to-frame deltas (although memory bandwidth may be an issue for the latter - maybe line-to-line?).

If FPGA space is getting tight I'd be inclined to ditch the microblaze out of the FPGA and use an extarnal MCU - the bandwidth to the MCU is going to be relatively minimal, so a simple SPI interface to something like an AVR or ARM should be more than adequate. It would also make it easier for others wanting to build their own & play with the software - Is it still the case that Xilinx charge for the Microblaze EDK? Even if you have to supply the FPGA stuff ready-made if the synth tools needed aren't readily availble, the software is probably what most people will want to play with..

Totally agree about colour - lighting high-speed stuff is hard enough without losing 2/3+ of it through the filters.
If anything, with a sensor as cheap (relatively..) as this, I'd be more tempted to build a dual-sensor system to get stereoscopic & multiple viewpoints. (how about a HS camera array for Matrix-style effects of fast events - this would be truly spectacular, and possibly something that nobody has done, even commercially..!)

Just wondering - did this project help you get the job? This is the sort of thing that ought to really impress an employer as it shows a wide variety of useful practical skills that are rare in the avarage college-leaver...

Cypress also makes a much larger sensor, 1280x1024 @ 450fps, but the FPGA dev board I have can't support the 160 bit data bus that would come off the 16 ADCs. I may try that sensor in a later project.

Just looked at the data for this - looks a pig to drive - it has no onboard ADCs so you'd need to hook 16 40msps ADCs to the outputs, and the O/P impedance is quite high so this would be an insane PCB layout task.
Take a look at the Micron MT9M413C36STM - same res, faster, on-chip ADCs, and cheaper. Stocked by Digikey.

tesla500

Sat Mar 08 2008, 07:47AM

Registered Member #347 Joined: Sat Mar 25 2006, 08:26AM
Location: Vancouver, Canada
Posts: 106

Hi Mike, thanks!

I will definitely make some more sensor PCBs. Several people have asked for them, and it would be nice to have a couple or three of these cameras to get different frame rates and angles on events. I looked at various development boards, but anything that had more RAM than this, or better yet a standard RAM slot, cost about $2k. It would probably be best to just make a PCB with everything on it, with a proper RAM slot.

Real-time compression may be practical with a big enough FPGA, either with a simple algorithm, or full fledged MPEG. The data rate on this is similar to that of HDTV, so it seems reasonable that it could be done. 720P HDTV is about 60 megapixels per second, this sensor does about 80 megapixels per second. Although, given how cheap RAM is nowadays, compression seems less and less necessary.

The Microblaze is definitely way overkill for this application. During record all it really does is advance the frame write pointer when the sensor is done writing a frame. I tested implementing the design with the Microblaze commented out, and it took up about 20% instead of 76% of the FPGA. Xilinx still does charge for EDK, but the compiler is open source, so it may be possible to program it without having EDK.

The 8 bit Picoblaze would be more appropriate, but it lacks proper dev tools, I believe you need to reprogram the entire FPGA to change the code. The Picoblaze dev tools are free, including the C complier.

For stereoscopic/matrix effects, it would probably be best to make provisions to sync multiple cameras together, maybe with variable phasing. That would definitely lead to some very interesting shots, being able to "move" the camera around at otherwise impossible speeds.

This project may have helped me get the job, but I think it was mainly knowing some employees in the company through the local electric vehicle club. I have a friend who graduated about the same time as me, and he builds hobby projects at only a slightly lower level than this. He searched for six months and didn't find a related job, so it seems that employers really want a degree or job experience, rather than a diploma or hobby experience.

I have looked at the Micron MT9M413C36STM, but a major limitation is that you can only reduce the Y resolution to increase frame rate, because it has one ADC on each column. You can reduce both X and Y resolution on the Cypress sensors. For example, if you want 7,000fps, on the Micron sensor you'd have to run about 1280x72, while on the Cypress one you could run 320x240.

I recently saw a news release from Cypress saying they're going to release an upgraded version of the LUPA-1300 that will have internal ADCs and 12 LVDS outputs. I would defiantly wait for that one before attempting anything with the higher resolution sensors.

I was wondering, with your Kodak camera, is there significant fixed-pattern noise? Correcting the noise isn't difficult in a modern system, but it would have been extremely difficult on one that recorded directly to analog tape.

David

previous 1 2 3 4 next

Moderator(s): Chris Russell, Noelle, Alex, Tesladownunder, Dave Marshall, Dave Billington, Bjørn, Steve Conner, Wolfram, Kizmo, Mads Barnkob