FPGA are excellent for image processing, their parallel nature easily allows us to implement parallel processing stages. Over the years we have done several image processing projects including
All of these systems have been created with devices within the Cost Optimised Portfolio, for the application I am going to create a base design which is based on one a slightly larger FPGA. I am doing this as I want to be able to demonstrate more complex implementations of algorithms for example dark field subtraction, pixel correction, object tracking and more advanced algorithms.
I am going to use a pure FPGA as I want to also explore the use of MicroBlaze V in the control plane for more complex applications.
For this platform I am going to create use the Kintex 7 based Genesys 2 development board which provides me with HDMI Input and Output interfaces. These HDMI interfaces will be ideal for capturing images from a sports camera or test equipment and displaying images on a screen.
To get started with I am going to create a simple pass through design, this will receive the data over HDMI, convert it from parallel video format with vertical and horizontal syncs etc into a AXI Stream.
I want to convert it to an AXI stream as this is a standard interface used by most image processing blocks.
To output the data I will be using a AXI Stream to Video out module which generates parallel video under the control of a video timing generator.
This video timing generator, and eventual image processing blocks will be controlled and configured by a MicroBlaze V. This is the latest MicroBlaze processor which is based on the open source RISC-V Instruction Set Architecture.
Unlike many of my previous example were a VDMA was used for this base application there will be no VMDA. This will generate a solution which provides the minimum latency between input and output.
I did however select a board which has 1GB of DDR3 should one of the algorithms need to be used for image buffering etc.
The design in Vivado can be split into to sections the image processing pipeline which contains the following IP cores
DVI2RGB – Digilent IP core for DVI conversion to RGB format
Video in to AXI Stream – Vivado IP block to convert RGB video to AXI Stream format
AXI Stream to Video out – Vivado IP block to convert AXI Stream to format to a RGB Format.
Video Timing Controller – Vivado IP block configured to both detect the timing received and generate the timing for the output. This will allow us later to support VDMA applications if needed.
AXI Stream FIFO – Vivado IP Block configured in packet mode to buffer a line before passing it through.
To aid with timing closure several AXIS register slices are inserted within the image processing pipeline.
The MicroBlaze V controller subsystem, comprises a MicroBlaze V configured as a microcontroller. This configuration enables both the AXI peripheral data and instruction interfaces, both of these are connected to a AXI Interconnect.
The AXI Interconnect is connected to the following IP blocks
UartLite – Vivado IP Block to provide a UART console
AXI GPIO – Vivado IP Block, monitors the display hot plug detect and camera hot plug detect output.
MIG 7 Series – Vivado IP Block providing the interface to the DDR3 on the Genesys2 board.
Processor Reset Block – Vivado IP Block controls the reset of the system
AXI Interrupt Controller – Vivado IP Block provides interrupts to the MicroBlaze Processor.
The design also contains a MicroBlaze V Debug Module, which is a Vivado IP Block enabling debugging of the MicroBlaze V using Vitis.
Clocking wise the MIG is provided with the board differential clock which is at 200 MHz. The MIG generates a UI clock at 100 MHz which reflects the 4:1 clocking scheme used by the MIG. An additional UI clock is also generated at 200 MHz, this is the reference clock for the RGB2DVI module.
I have configured this for 720P as that is that the sports camera I have provides as its output. So the AXI Stream clock will run at 150 MHz as that is twice the clock frequency of the pixel clock.
The software side of this is simple as we hope it would be all we need to do is configure the video timing generator with the video output mode. It also asserts the HPD for the camera.
#include <stdio.h>
#include "platform.h"
#include "xil_printf.h"
#include "xvtc.h"
#include "xgpio.h"
#include "vga.h"
#include "xparameters.h"
XVtc VtcInst;
XVtc_Config *vtc_config ;
XGpio hpd_in;
XVtc_SourceSelect SourceSelect;
int main()
{
u16 result;
VideoMode video;
XVtc_Timing vtcTiming;
init_platform();
printf("Setting up VTC\n\r");
vtc_config = XVtc_LookupConfig(XPAR_XVTC_0_BASEADDR);
XVtc_CfgInitialize(&VtcInst, vtc_config, vtc_config->BaseAddress);
//configure and assert the HPD
XGpio_Initialize(&hpd_in, XPAR_XGPIO_0_BASEADDR);
XGpio_DiscreteWrite(&hpd_in,1,0x1);
sleep(20);
XGpio_DiscreteWrite(&hpd_in,2,0x1); ///needs time here
video = VMODE_1280x720;
vtcTiming.HActiveVideo = video.width; /**< Horizontal Active Video Size */
vtcTiming.HFrontPorch = video.hps - video.width; /**< Horizontal Front Porch Size */
vtcTiming.HSyncWidth = video.hpe - video.hps; /**< Horizontal Sync Width */
vtcTiming.HBackPorch = video.hmax - video.hpe + 1; /**< Horizontal Back Porch Size */
vtcTiming.HSyncPolarity = video.hpol; /**< Horizontal Sync Polarity */
vtcTiming.VActiveVideo = video.height; /**< Vertical Active Video Size */
vtcTiming.V0FrontPorch = video.vps - video.height; /**< Vertical Front Porch Size */
vtcTiming.V0SyncWidth = video.vpe - video.vps; /**< Vertical Sync Width */
vtcTiming.V0BackPorch = video.vmax - video.vpe + 1;; /**< Horizontal Back Porch Size */
vtcTiming.V1FrontPorch = video.vps - video.height; /**< Vertical Front Porch Size */
vtcTiming.V1SyncWidth = video.vpe - video.vps; /**< Vertical Sync Width */
vtcTiming.V1BackPorch = video.vmax - video.vpe + 1;; /**< Horizontal Back Porch Size */
vtcTiming.VSyncPolarity = video.vpol; /**< Vertical Sync Polarity */
vtcTiming.Interlaced = 0;
memset((void *)&SourceSelect, 0, sizeof(SourceSelect));
SourceSelect.VBlankPolSrc = 1;
SourceSelect.VSyncPolSrc = 1;
SourceSelect.HBlankPolSrc = 1;
SourceSelect.HSyncPolSrc = 1;
SourceSelect.ActiveVideoPolSrc = 1;
SourceSelect.ActiveChromaPolSrc= 1;
SourceSelect.VChromaSrc = 1;
SourceSelect.VActiveSrc = 1;
SourceSelect.VBackPorchSrc = 1;
SourceSelect.VSyncSrc = 1;
SourceSelect.VFrontPorchSrc = 1;
SourceSelect.VTotalSrc = 1;
SourceSelect.HActiveSrc = 1;
SourceSelect.HBackPorchSrc = 1;
SourceSelect.HSyncSrc = 1;
SourceSelect.HFrontPorchSrc = 1;
SourceSelect.HTotalSrc = 1;
XVtc_RegUpdateEnable(&VtcInst);
XVtc_SetGeneratorTiming(&VtcInst, &vtcTiming);
XVtc_SetSource(&VtcInst, &SourceSelect);
XVtc_EnableGenerator(&VtcInst);
XVtc_Enable(&VtcInst);
XVtc_EnableDetector(&VtcInst);
XVtc_Enable(&VtcInst);
xil_printf("Video Mode = %i ", result);
xil_printf("\n\r");
printf("VTC Set Up\n\r");
cleanup_platform();
return 0;
}Once a camera is connected to the board we should then see images passing through.
When it comes to resource utilisation the logic footprint is minimal which reserves lots for experimentation.
The project is available on my github, instructions on how to restore it are here, now we have a platform we can start exploring.
Workshops and Webinars
If you enjoyed the blog why not take a look at the free webinars, workshops and training courses we have created over the years. Highlights include
Professional PYNQ Learn how to use PYNQ in your developments
Introduction to Vivado learn how to use AMD Vivado
Ultra96, MiniZed & ZU1 three day course looking at HW, SW and PetaLinux
Arty Z7-20 Class looking at HW, SW and PetaLinux
Mastering MicroBlaze learn how to create MicroBlaze solutions
HLS Hero Workshop learn how to create High Level Synthesis based solutions
Perfecting Petalinux learn how to create and work with PetaLinux OS
Boards
Get an Adiuvo development board
Adiuvo Spartan 7 / RPi 2040 Embedded System Development Board
Embedded System Book
Do you want to know more about designing embedded systems from scratch? Check out our book on creating embedded systems. This book will walk you through all the stages of requirements, architecture, component selection, schematics, layout, and FPGA / software design. We designed and manufactured the board at the heart of the book! The schematics and layout are available in Altium here Learn more about the board (see previous blogs on Bring up, DDR validation, USB, Sensors) and view the schematics here.
Comments