Hi all,
I am looking for developing an IP camera based system based on STMicroelectronics STM32. The way it is supposed to work is:
STM32F4 will have few IP cameras connected to it. It may have a web server running so it can be accessed from outside.
We will have a mobile app running on a mobile device / tablet. It should be able to connect to a particular IP camera through the web server (or any other mechanism).
End result expected is as a user if I run the app on my device, I should be able to connect to a particular site (say my home) and get the live feed of a camera.
I do not need to do any image processing.
Now the questions:
1. Is STM32F4 suitable for this application?
2. What other components I am missing in the overall system?
3. Using a storage media with STM, such as a SD card, I can record the video and then send it when requested. Does video recording need knowledge of the protocol used by the camera (say H.264 compression method etc.)?
4. Any pointers where I can start studying this?
Thanks a lot,
Gopal
Hi Gopal, with Embedded World running this week, a few of our embedded experts are enjoying Nuremberg!
What do you want the MCU to do with these video streams and what kind of throughput (image size/quality/framerate) would do you expect?
The higher end STM32 F4 has one camera and Ethernet interfaces: STM32F4 Series - STMicroelectronics but these stays microcontrollers... (having said that, I just have to say it's not possible for Jens can run HD video on them... )
You might benefit from the comfort of running a fuller operating system... You would also be able to reuse code/libraries as well and wouldn't have to think too much about task scheduling.
I like your idea.
Alban
To me, it seems like the STM32F427 could do it (as Alban mentions).
(Yes, the STM32F417 do have a camera interface, but the STM32F427 have the Chrom-ART in addition, which I think you will very much like to have).
I've already seen a 'live demo' showing camera-to-LCD display video. The OV76xx cameras deliver up to 640x480 @ 30 FPS.
Let's examine that a bit.
Some of the OV76 cameras come with an on-board RAM cache. You want one of those, as it can significantly reduce the demand on your MCU.
If using a OV7670 camera, it is of course not straight-forward to find out the best resolution it can give you, when looking at the datasheet.
The datasheet promises the following: 16-bit RGB pixels. 640x480 maximum resolution and 30 FPS maximum update frequency.
This does not mean you get a 16-bit 640x480@30Hz video. Looking deeper into the datasheet, it does seem to promise a 15 FPS 640x480 YUV, where this YUV is 4:2:2 (8-bit).
It also promises a 30 FPS VGA (which again is 640x480) in YUV. Now, that's 640x480@30FPS in 8-bit color. -I see no promise of a 16-bit color in VGA at 30 FPS, though.
But usiing those numbers, we can calculate the number of bytes you need to receive per second:
640 x 480 x 30 x 8 / 8 = 9216000 Bytes/sec = 9000 KBytes/sec = 8.79 MB/sec.
Most SD/MMC cards would be able to handle a transfer like this. See Secure Digital - Wikipedia, the free encyclopedia for details.
-So the first thing to do might be to capture the uncompressed video stream and store it directly to the SD card. Your STM32F427 already has a hardware SDIO interface, which supports 1-bit, 4-bit and 8-bit interfaces.
The STM32F427 has the following features you'll benefit from:
If transferring this video stream over Ethernet, you need a Fast Ethernet connection, that is 100 Mbit/sec. In fact, you should go for MII, not RMII, because RMII's transfer speed is 10 MB/sec, whereas MII is 12.5 MB/sec.
As 128 KB RAM is not much, when you're dealing with video-streams, I am sure you will want to interface an external SDRAM or perhaps an external SRAM to your MCU.
Because you're dealing with fairly high speeds, you'll have to design a PCB carefully. This means: Ground-planes and meanders (traces in a family need to be the same lengths). Ground planes are more important than meanders, but meanders are important - so make sure you study these topics. There's more regarding good designs on high-speed traces and memory interfacing (another way of saying that I don't have all the answers myself).
If you can find a SDRAM, which is fairly low-cost (say $5) and gives you something like 32 MB RAM or more (32 MB = 256Mbit), I would recommend that.
(I've seen a 64 MB chip for less than $7 on eBay. I'd like to warn you against buying from eBay, because most sellers do not know to handle components in an ESD safe way. One seller, who do know how to handle components correctly, is Happy Nicky, and I would like to recommend him - if you need a specific SDRAM for instance, please provide him with all the details, he will usually be able to find some chips which conforms to those specifications).
SD card interface is easy. Connect a 33K pull-up on each of the data lines, a 4.7uF+100nF cap on VCC, and you should be OK. Provide VCC through a ferrite bead to reduce noise.
You can use Chrom-ART for cropping the image (eg. remove flickering borders), converting resolutions and other graphic operations, such as blending and overlaying other images. Perhaps you want rounded corners in your video-stream; Chrom-ART would be able to do that by blending a rounded-corner image with the video image - and faster than assembly code could do the same operation.
Here's a great article on interfacing a camera with the STM32F4: Embedded Programmer: Hacking the OV7670 camera module (SCCB cheat sheet inside)
Thanks jensbauer
It is very helpful (as always) and detailed reply.
I do not want to do any video processing, neither show the video on my system. This is a kind of intrusion alarm system where, a sensor connected to the system triggers and then the system wants to send either of the following data to a remote server.
1) Video recording for a few seconds post event and may be a few seconds pre-event. So this implies that the video should be recorded continuously and may be overwritten say every 10 seconds or so.
2) Or the system may be configured to send few images to the server. Again pre-event and post-event.
The cameras would be bought by the end user of the system, not a design time decision. So the best would be IP cameras available off the shelf. It could be multiple IP cameras installed for a single system. All the IP cameras would be connected to the system via LAN. The system will identify the right camera to trigger by its IP address (Is that possible? I don't know yet).
So you explanation of having more RAM and a SD card to store uncompressed video looks quite suitable. I think if DMA is setup then ethernet to SD card data can be directly transferred. (Again, I have not done this yet, so unsure).
I have seen some cameras offering on-board storage with a SD card. So you may just pull the images from the camera.
Also some cameras offer H.264 like encoding as well.
albanrampon,
A full fledged OS will be very good but am trying to research out if I can do without that. Just running a normal RTOS (Keil RTX) and trying to explore the possibilities.
Thanks for your inputs...
Thank you for the kind words.
If you need to connect multiple cameras, you'll probably need at least a Cortex-M4, perhaps something even faster.
But if you choose a single camera per Cortex-M4, I think it would be possible, and probably also more secure, in case the burglar discovers the recorder (note: Such people don't always play by the rules - they are very clever; I just wish they would use their skills for developing ARM based gadgets instead).
Transferring data from Ethernet directly to the SD-card ... Well, ehm, possible (by DMA-to-buffer-to-DMA), but you'll get data in random order, and you will probably have problems with reading a raw SD-card on a desktop computer.
It would probably be best to transfer the Ethernet data to a buffer, There won't really be much overhead time-wise, because you'll sort packets and "insert" data for the FAT32.
Please note: The FAT32 implementations available on the net are fragile and contains a bunch of errors; use with caution!
-If you're going to use them, I advice you to go through them line-by-line and understand each piece of the code and what's supposed to be there.
You could perhaps use the camera as a sensor.
If the image changes since last time, an event is starting (note: you're capturing the image anyway).
-However, if the cameras are recording through a window, you'll get a lot of false positives from the tree right outside the house.
If the camera offers some kind of compression, the length of the data usually varies when an image changes, so that could be a way of detecting changes easily.
One other thing to remember: SPI allows you to share the same lines for multiple slaves.
That means: You can "stack" your SD-card connectors, just connecting a different GPIO pin for the "CS" pin.
The SDIO interface does not allow stacking... BUT! There is a workaround:
Using an AND gate on the CS pin, you can connect the SD_CS from the MCU to one of the inputs on the AND gate. The output is connected to the SD/MMC card socket. The other input on the AND gate is connected to your GPIO pin, and now you're allowed to use GPIO pins to select which card you want to read from/write to.
Of course: You can only access one card at a time, but this method allows you to save data on more than one "low-cost" 32GB card.
Let's check the recording length of your uncompressed video from a single camera. Using the calculation I mentioned in the earlier post, we'll assume 9MB/sec:
32GB = 32 * 1024 MB = 32768 MB.
32768 MB/9 MB per sec = 3640 seconds.
3640 seconds / 60 seconds per minute = 60 minutes = 1 hour.
It may be a good idea to split the recorded files, so they will always be less than 1 GB, otherwise you'll get in trouble with some operating systems (did anyone say DVD?)
Yes, it is going to be Cortex-M4F.
Ok, I never worked with DMA so missed this point of "out-of-order" packets.
Using camera as sensor is also good idea. Probably I can use that in addition to the motion sensors.
At the moment, the exploration is based on the need for having multiple cameras per CortexM4. But if that is *not* possible, then single camera is going to be a compromise.
But from the calculations, it looks feasible, doesn't it?
By the way, I superlike your saying
jensbauer wrote: (note: Such people don't always play by the rules - they are very clever; I just wish they would use their skills for developing ARM based gadgets instead).
jensbauer wrote:
(note: Such people don't always play by the rules - they are very clever; I just wish they would use their skills for developing ARM based gadgets instead).
The DMA is faithful, but the packets from Ethernet might come in any order, as they can be disturbed by a switch, a router and in general be lost, so they would need to be sent again. (IP protocol).
Due to that some packets will probably need re-transmission, it's good to have a little more bandwidth than the "exact match", so 12.5 MB/sec would probably be good for a 9MB/sec transfer.
-So yes, if not cutting things too close to the limits (keeping some free air / "breathing-room" around everything) should help taking such things into account.
Did you made this camera ONVIF compatible?. Did you use RTOS or raw API?
I am also trying to make a ONVIF conformant surveillance camera with STM32H743? Is it possible to make such cameras in controllers.