Loading repository data…
Loading repository data…
BoChen-Ye / repository
This is my hobby project with System Verilog to accelerate LeViT Network which contain CNN and Attention layer.
A transparent discovery signal based on current public GitHub metadata.
This score does not audit code, security, maintainers, documentation quality, or suitability. Verify the repository and its current documentation before adoption.
This is my hobby project by using System Verilog. I worked on this project continuously for about 4-5 weeks, only dedicating evenings and weekends to it. I wrote approximately 2000+ lines of SV design code and 1900+ testbench code in total. It will cost 97000+ LUT and 51000+ FF in FPGA implementation(Vivado).
This work is based on LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference.
But I've simplified the original large network into a smaller one, so I call it Tiny_LeViT.
All design file are in src folder and testbench in sim folder, include:
Completed:
Of course, due to the complexity of hardware accelerators and network functions, many problems arise during the simplification process, some of which are yet to be resolved or will be addressed in the future.
Problem:
Anyway, if you have questions or improvements, just tell me. Thanks!
src/Tiny_LeViT_top.sv is the top level of Tiny_LeViT. It contain three convolutional layer(16,8,4), four stage1(2-head attention and MLP) and stage2(4-head attention and MLP). In the end, it has e average pooling module.src/definition.sv.end which indicate output result and en which enable the module.src/Conv_core_sa.sv and src/PE_ROW_SystolicArry.sv is normal version of convolution layer which delay is only 3 cycle from input data to first output data.src/Conv16_core.sv: 10 cycle, src/Conv8_core.sv: 6 cycle,src/Conv4_core.sv: 4 cycle.src/Stage_2head.sv have 2 attention core and 1 MLP.src/Stage_4head.sv have 4 attention core and 1 MLP.i_up and i_bo indicate the two input(up and down).cout is the result and rem is remainder.