IP Camera interface via STM32

Hi all,

I am looking for developing an IP camera based system based on STMicroelectronics STM32. The way it is supposed to work is:

STM32F4 will have few IP cameras connected to it. It may have a web server running so it can be accessed from outside.

We will have a mobile app running on a mobile device / tablet. It should be able to connect to a particular IP camera through the web server (or any other mechanism).

End result expected is as a user if I run the app on my device, I should be able to connect to a particular site (say my home) and get the live feed of a camera.

I do not need to do any image processing.

Now the questions:

1. Is STM32F4 suitable for this application?

2. What other components I am missing in the overall system?

3. Using a storage media with STM, such as a SD card, I can record the video and then send it when requested. Does video recording need knowledge of the protocol used by the camera (say H.264 compression method etc.)?

4. Any pointers where I can start studying this?

Thanks a lot,

Gopal

Parents
  • To me, it seems like the STM32F427 could do it (as Alban mentions).

    (Yes, the STM32F417 do have a camera interface, but the STM32F427 have the Chrom-ART in addition, which I think you will very much like to have).

    I've already seen a 'live demo' showing camera-to-LCD display video. The OV76xx cameras deliver up to 640x480 @ 30 FPS.

    Let's examine that a bit.

    Some of the OV76 cameras come with an on-board RAM cache. You want one of those, as it can significantly reduce the demand on your MCU.

    If using a OV7670 camera, it is of course not straight-forward to find out the best resolution it can give you, when looking at the datasheet.

    The datasheet promises the following: 16-bit RGB pixels. 640x480 maximum resolution and 30 FPS maximum update frequency.

    This does not mean you get a 16-bit 640x480@30Hz video. Looking deeper into the datasheet, it does seem to promise a 15 FPS 640x480 YUV, where this YUV is 4:2:2 (8-bit).

    It also promises a 30 FPS VGA (which again is 640x480) in YUV. Now, that's 640x480@30FPS in 8-bit color. -I see no promise of a 16-bit color in VGA at 30 FPS, though.

    But usiing those numbers, we can calculate the number of bytes you need to receive per second:

    640 x 480 x 30 x 8 / 8 = 9216000 Bytes/sec = 9000 KBytes/sec = 8.79 MB/sec.

    Most SD/MMC cards would be able to handle a transfer like this. See Secure Digital - Wikipedia, the free encyclopedia for details.

    -So the first thing to do might be to capture the uncompressed video stream and store it directly to the SD card. Your STM32F427 already has a hardware SDIO interface, which supports 1-bit, 4-bit and 8-bit interfaces.

    The STM32F427 has the following features you'll benefit from:

    • Chrom-ART
    • Ethernet (10/100)
    • Camera Interface
    • SDIO
    • Interface for external SDRAM and SRAM
    • 128KByte on-chip SRAM which can be accessed by the DMA (reserve this exclusively for the DMA)
    • 64KByte additional CCMRAM (which cannot be accessed by the DMA, but let the CPU use this for calculations, quick data storage and stack)

    If transferring this video stream over Ethernet, you need a Fast Ethernet connection, that is 100 Mbit/sec. In fact, you should go for MII, not RMII, because RMII's transfer speed is 10 MB/sec, whereas MII is 12.5 MB/sec.

    As 128 KB RAM is not much, when you're dealing with video-streams, I am sure you will want to interface an external SDRAM or perhaps an external SRAM to your MCU.

    Because you're dealing with fairly high speeds, you'll have to design a PCB carefully. This means: Ground-planes and meanders (traces in a family need to be the same lengths). Ground planes are more important than meanders, but meanders are important - so make sure you study these topics. There's more regarding good designs on high-speed traces and memory interfacing (another way of saying that I don't have all the answers myself).

    If you can find a SDRAM, which is fairly low-cost (say $5) and gives you something like 32 MB RAM or more (32 MB = 256Mbit), I would recommend that.

    (I've seen a 64 MB chip for less than $7 on eBay. I'd like to warn you against buying from eBay, because most sellers do not know to handle components in an ESD safe way. One seller, who do know how to handle components correctly, is Happy Nicky, and I would like to recommend him - if you need a specific SDRAM for instance, please provide him with all the details, he will usually be able to find some chips which conforms to those specifications).

    SD card interface is easy. Connect a 33K pull-up on each of the data lines, a 4.7uF+100nF cap on VCC, and you should be OK. Provide VCC through a ferrite bead to reduce noise.

    You can use Chrom-ART for cropping the image (eg. remove flickering borders), converting resolutions and other graphic operations, such as blending and overlaying other images. Perhaps you want rounded corners in your video-stream; Chrom-ART would be able to do that by blending a rounded-corner image with the video image - and faster than assembly code could do the same operation.

    Here's a great article on interfacing a camera with the STM32F4: Embedded Programmer: Hacking the OV7670 camera module (SCCB cheat sheet inside)

Reply
  • To me, it seems like the STM32F427 could do it (as Alban mentions).

    (Yes, the STM32F417 do have a camera interface, but the STM32F427 have the Chrom-ART in addition, which I think you will very much like to have).

    I've already seen a 'live demo' showing camera-to-LCD display video. The OV76xx cameras deliver up to 640x480 @ 30 FPS.

    Let's examine that a bit.

    Some of the OV76 cameras come with an on-board RAM cache. You want one of those, as it can significantly reduce the demand on your MCU.

    If using a OV7670 camera, it is of course not straight-forward to find out the best resolution it can give you, when looking at the datasheet.

    The datasheet promises the following: 16-bit RGB pixels. 640x480 maximum resolution and 30 FPS maximum update frequency.

    This does not mean you get a 16-bit 640x480@30Hz video. Looking deeper into the datasheet, it does seem to promise a 15 FPS 640x480 YUV, where this YUV is 4:2:2 (8-bit).

    It also promises a 30 FPS VGA (which again is 640x480) in YUV. Now, that's 640x480@30FPS in 8-bit color. -I see no promise of a 16-bit color in VGA at 30 FPS, though.

    But usiing those numbers, we can calculate the number of bytes you need to receive per second:

    640 x 480 x 30 x 8 / 8 = 9216000 Bytes/sec = 9000 KBytes/sec = 8.79 MB/sec.

    Most SD/MMC cards would be able to handle a transfer like this. See Secure Digital - Wikipedia, the free encyclopedia for details.

    -So the first thing to do might be to capture the uncompressed video stream and store it directly to the SD card. Your STM32F427 already has a hardware SDIO interface, which supports 1-bit, 4-bit and 8-bit interfaces.

    The STM32F427 has the following features you'll benefit from:

    • Chrom-ART
    • Ethernet (10/100)
    • Camera Interface
    • SDIO
    • Interface for external SDRAM and SRAM
    • 128KByte on-chip SRAM which can be accessed by the DMA (reserve this exclusively for the DMA)
    • 64KByte additional CCMRAM (which cannot be accessed by the DMA, but let the CPU use this for calculations, quick data storage and stack)

    If transferring this video stream over Ethernet, you need a Fast Ethernet connection, that is 100 Mbit/sec. In fact, you should go for MII, not RMII, because RMII's transfer speed is 10 MB/sec, whereas MII is 12.5 MB/sec.

    As 128 KB RAM is not much, when you're dealing with video-streams, I am sure you will want to interface an external SDRAM or perhaps an external SRAM to your MCU.

    Because you're dealing with fairly high speeds, you'll have to design a PCB carefully. This means: Ground-planes and meanders (traces in a family need to be the same lengths). Ground planes are more important than meanders, but meanders are important - so make sure you study these topics. There's more regarding good designs on high-speed traces and memory interfacing (another way of saying that I don't have all the answers myself).

    If you can find a SDRAM, which is fairly low-cost (say $5) and gives you something like 32 MB RAM or more (32 MB = 256Mbit), I would recommend that.

    (I've seen a 64 MB chip for less than $7 on eBay. I'd like to warn you against buying from eBay, because most sellers do not know to handle components in an ESD safe way. One seller, who do know how to handle components correctly, is Happy Nicky, and I would like to recommend him - if you need a specific SDRAM for instance, please provide him with all the details, he will usually be able to find some chips which conforms to those specifications).

    SD card interface is easy. Connect a 33K pull-up on each of the data lines, a 4.7uF+100nF cap on VCC, and you should be OK. Provide VCC through a ferrite bead to reduce noise.

    You can use Chrom-ART for cropping the image (eg. remove flickering borders), converting resolutions and other graphic operations, such as blending and overlaying other images. Perhaps you want rounded corners in your video-stream; Chrom-ART would be able to do that by blending a rounded-corner image with the video image - and faster than assembly code could do the same operation.

    Here's a great article on interfacing a camera with the STM32F4: Embedded Programmer: Hacking the OV7670 camera module (SCCB cheat sheet inside)

Children