Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fpga): support PCIe XDMA based difftest #453

Merged
merged 41 commits into from
Jan 10, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
57ea5f7
fpga: Add pcie XDMA framework
xiaokamikami Aug 27, 2024
6ab769b
fpga: add mpool
xiaokamikami Aug 28, 2024
e989cc1
difftest: Move mempool to common code
xiaokamikami Aug 28, 2024
25cd18b
fpga: Add function of difftest through dma interface
xiaokamikami Aug 29, 2024
c20ef8d
fpga: add independent compilation and usage support under fpga
xiaokamikami Aug 29, 2024
2511ef7
fpga: modify the xdma initi process
xiaokamikami Sep 18, 2024
f9e59bc
difftest: Fixed an issue where the block structure was not memory safe
xiaokamikami Sep 20, 2024
3ca0542
fpga: The memory pool with sliding window was added to realize multi-…
xiaokamikami Sep 23, 2024
a82ec95
fpga: fix mpool format
xiaokamikami Sep 23, 2024
0804440
fpga: modify xdma to be multi-channel configurable and use a sliding …
xiaokamikami Sep 23, 2024
e047edc
fpga: Improve the operation logic of fpga diff
xiaokamikami Sep 24, 2024
7686245
fpga: Remove redundant mempool-MemoryBlock definitions
xiaokamikami Sep 25, 2024
744d519
fpga: The adaptation pack processes the new batch
xiaokamikami Sep 25, 2024
9a976d5
CI: add fpga-diff compile ci
xiaokamikami Sep 25, 2024
9c51fdc
fpga: svdpi.h is not referenced when fpga is used
xiaokamikami Sep 26, 2024
97ab23c
fpga: Burn workload to fpga ddr at boot time
xiaokamikami Sep 26, 2024
4effd8c
fpga: Load memory for the ref module
xiaokamikami Sep 27, 2024
71a9032
fpga: The optimal parameters of mpool are adjusted
xiaokamikami Nov 1, 2024
4e7deac
fpga: batch processing on fpga diff data is used by default
xiaokamikami Nov 4, 2024
15bbadb
fpga: remove the redundant include and disable DB on fpga
xiaokamikami Nov 4, 2024
f5986f3
fpga: fix formate
xiaokamikami Nov 4, 2024
c991c3d
fpga: fix a stuck startup bug
xiaokamikami Nov 6, 2024
327d135
fpga: Add idx verification for dma packets and initiate process repai…
xiaokamikami Nov 8, 2024
6d38a2e
fpga: mpool fix computation of free_num
xiaokamikami Nov 11, 2024
76be72c
difftest: Turn off h2c when it is not needed
xiaokamikami Nov 18, 2024
7a346c8
difftest: Added packet parsing under squash for XDMA
xiaokamikami Dec 16, 2024
0ec4128
fpga: The packet adapted to squash format runs FPGA-diff
xiaokamikami Dec 17, 2024
25b8b9a
WIP: expose IO for FPGA no batch
klin02 Dec 12, 2024
9ed3eb6
fpga: fix squash packge memory sequence
xiaokamikami Jan 4, 2025
51e9b9f
fpga: fix suqash struct memory alignment,Support for switching parsi…
xiaokamikami Jan 4, 2025
1208b38
fpga: The running parameter max-instrs is modified from hexadecimal t…
xiaokamikami Jan 6, 2025
80efb16
fpga: fix fpga_mian init process
xiaokamikami Jan 6, 2025
0019b48
fpga: It supports running the difftest in squash mode on fpga difftes…
xiaokamikami Jan 7, 2025
63cc888
fpga: rename **unpack.cpp
xiaokamikami Jan 8, 2025
e5065b3
fpga: fix Squash typedef
xiaokamikami Jan 8, 2025
119990c
CI: enable fpga-diff compile ci
xiaokamikami Jan 8, 2025
f850fb2
CI: fix CI compile
xiaokamikami Jan 8, 2025
8b33081
CI: tidy up the file code format
xiaokamikami Jan 9, 2025
df0e48a
fpga: Clean up the code, remove the hardware changes, and keep the so…
xiaokamikami Jan 9, 2025
282835a
fpga: edit makefile out path
xiaokamikami Jan 9, 2025
be80b04
fpga: merge macro variables and get_load_img_size move get_img_size
xiaokamikami Jan 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -306,3 +306,32 @@ jobs:
make simv VCS=verilator -j2
./build/simv +workload=$WORKLOAD +b=0 +e=-1 +diff=$REF_SO +max-instrs=5000 +warmup_instr=1000
make clean

test-difftest-fpga:
poemonsense marked this conversation as resolved.
Show resolved Hide resolved
runs-on: ubuntu-22.04

needs: test-difftest-main

steps:
- uses: actions/checkout@v4

- name: Prepare environment
run: |
cd $GITHUB_WORKSPACE/..
wget https://github.com/OpenXiangShan/xs-env/raw/refs/heads/master/install-verilator.sh
wget https://github.com/OpenXiangShan/xs-env/raw/refs/heads/master/setup-tools.sh
sudo bash setup-tools.sh

- name: Prepare NutShell
run: |
cd $GITHUB_WORKSPACE/..
git clone -b dev-difftest --single-branch https://github.com/OSCPU/NutShell.git
cd NutShell && rm -rf difftest && cp -r $GITHUB_WORKSPACE .
echo "NOOP_HOME=$(pwd)" >> $GITHUB_ENV

- name: FPGA-difftest Build
run: |
cd $NOOP_HOME
make sim-verilog MILL_ARGS="--difftest-config SF" -j2
cd ./difftest
make fpga-build FPGA=1
10 changes: 9 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,13 @@ SIM_VSRC = $(shell find $(VSRC_DIR) -name "*.v" -or -name "*.sv")

# DiffTest support
DIFFTEST_CSRC_DIR = $(abspath ./src/test/csrc/difftest)
# FPGA-Difftest support
ifeq ($(FPGA),1)
$(info FPGA is enabled. ChiselDB and ConstantIn are implicitly disabled.)
WITH_CHISELDB = 0
WITH_CONSTANTIN = 0
endif

DIFFTEST_CXXFILES = $(shell find $(DIFFTEST_CSRC_DIR) -name "*.cpp")
ifeq ($(NO_DIFF), 1)
SIM_CXXFLAGS += -DCONFIG_NO_DIFFTEST
Expand Down Expand Up @@ -232,8 +239,9 @@ include verilator.mk
include vcs.mk
include palladium.mk
include libso.mk
include fpga.mk

clean: vcs-clean pldm-clean
clean: vcs-clean pldm-clean fpga-clean
rm -rf $(BUILD_DIR)

format: scala-format clang-format
Expand Down
20 changes: 20 additions & 0 deletions fpga.mk
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
FPGA_TARGET = $(BUILD_DIR)/fpga-host
FPGA_CSRC_DIR = $(abspath ./src/test/csrc/fpga)
FPGA_CONFIG_DIR = $(abspath ./config) # Reserve storage for xdma configuration

FPGA_CXXFILES = $(SIM_CXXFILES) $(shell find $(FPGA_CSRC_DIR) -name "*.cpp")
FPGA_CXXFLAGS = $(subst \\\",\", $(SIM_CXXFLAGS)) -I$(FPGA_CSRC_DIR) -DNUM_CORES=$(NUM_CORES) -DCONFIG_PLATFORM_FPGA -O2
FPGA_LDFLAGS = $(SIM_LDFLAGS) -lpthread -ldl

DMA_CHANNELS ?= 1
FPGA_LDFLAGS += -DCONFIG_DMA_CHANNELS=$(DMA_CHANNELS)

fpga-build: fpga-clean fpga-host

$(FPGA_TARGET): $(FPGA_CXXFILES)
$(CXX) $(FPGA_CXXFLAGS) $(FPGA_CXXFILES) -o $@ $(FPGA_LDFLAGS)

fpga-host: $(FPGA_TARGET)

fpga-clean:
poemonsense marked this conversation as resolved.
Show resolved Hide resolved
rm -f $(FPGA_TARGET)
167 changes: 167 additions & 0 deletions src/test/csrc/common/mpool.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
/***************************************************************************************
* Copyright (c) 2025 Beijing Institute of Open Source Chip (BOSC)
* Copyright (c) 2020-2025 Institute of Computing Technology, Chinese Academy of Sciences
*
* DiffTest is licensed under Mulan PSL v2.
* You can use this software according to the terms and conditions of the Mulan PSL v2.
* You may obtain a copy of Mulan PSL v2 at:
* http://license.coscl.org.cn/MulanPSL2
*
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
*
* See the Mulan PSL v2 for more details.
***************************************************************************************/
#include "mpool.h"

void MemoryPool::init_memory_pool() {
memory_pool.reserve(NUM_BLOCKS);
for (size_t i = 0; i < NUM_BLOCKS; ++i) {
memory_pool.emplace_back();
block_mutexes[i].unlock();
}
}

void MemoryPool::cleanup_memory_pool() {
cv_empty.notify_all();
cv_filled.notify_all();
memory_pool.clear();
}

void MemoryPool::unlock_thread() {
cv_empty.notify_all();
cv_filled.notify_all();
}

char *MemoryPool::get_free_chunk() {
page_head = (write_index++) & REM_NUM_BLOCKS;
{
std::unique_lock<std::mutex> lock(block_mutexes[page_head]);
cv_empty.wait(lock, [this] { return empty_blocks > 0; });
}

--empty_blocks;
block_mutexes[page_head].lock();
return memory_pool[page_head].data.get();
}

void MemoryPool::set_busy_chunk() {
memory_pool[page_head].is_free = false;
block_mutexes[page_head].unlock();
cv_filled.notify_one();
++filled_blocks;
}

const char *MemoryPool::get_busy_chunk() {
page_end = (read_index++) & REM_NUM_BLOCKS;
{
std::unique_lock<std::mutex> lock(block_mutexes[page_end]);
cv_filled.wait(lock, [this] { return filled_blocks > 0; });
}
--filled_blocks;
block_mutexes[page_end].lock();
return memory_pool[page_end].data.get();
}

void MemoryPool::set_free_chunk() {
memory_pool[page_end].is_free = true;
block_mutexes[page_end].unlock();
cv_empty.notify_one();
++empty_blocks;
}

// Cleaning up memory pools
void MemoryIdxPool::cleanupMemoryPool() {
cv_empty.notify_all();
cv_filled.notify_all();
}

// Write a specified free block of a free window
bool MemoryIdxPool::write_free_chunk(uint8_t idx, const char *data) {
size_t page_w_idx;
{
std::lock_guard<std::mutex> lock(offset_mutexes);

page_w_idx = idx + group_w_offset;
// Processing of winding data at the boundary
if (memory_pool[page_w_idx].is_free.load() == false) {
size_t this_group = group_w_idx.load();
size_t offset = ((this_group & REM_MAX_GROUPING_IDX) * MAX_IDX);
page_w_idx = idx + offset;
write_next_count++;
// Lookup failed
if (memory_pool[page_w_idx].is_free.load() == false) {
printf("This block has been written, and there is a duplicate packge idx %d\n", idx);
return false;
}
} else {
write_count++;
// Proceed to the next group
if (write_count == MAX_IDX) {
memcpy(memory_pool[page_w_idx].data.get(), data, 4096);
memory_pool[page_w_idx].is_free.store(false);
size_t next_w_idx = wait_next_free_group();
group_w_offset = (next_w_idx & REM_MAX_GROUPING_IDX) * MAX_IDX;
write_count = write_next_count;
write_next_count = 0;
return true;
}
}
memory_pool[page_w_idx].is_free.store(false);
}
memcpy(memory_pool[page_w_idx].data.get(), data, 4096);

return true;
}

void MemoryIdxPool::wait_mempool_start() {
std::unique_lock<std::mutex> lock(window_mutexes);
cv_filled.wait(lock);
}

bool MemoryIdxPool::read_busy_chunk(char *data) {
size_t page_r_idx = read_count + group_r_offset;
size_t this_r_idx = ++read_count;

if (this_r_idx == MAX_IDX) {
read_count = 0;
size_t next_r_idx = wait_next_full_group();
group_r_offset = ((next_r_idx & REM_MAX_GROUPING_IDX) * MAX_IDX);
}
if (memory_pool[page_r_idx].is_free.load() == true) {
printf("An attempt was made to read the block of free %zu\n", page_r_idx);
return false;
}

memcpy(data, memory_pool[page_r_idx].data.get(), 4096);
memory_pool[page_r_idx].is_free.store(true);
return true;
}

size_t MemoryIdxPool::wait_next_free_group() {
size_t free_num = empty_blocks.fetch_sub(1, std::memory_order_relaxed) - 1;
cv_filled.notify_all();
//Reserve at least two free blocks
if (free_num <= 2) {
std::unique_lock<std::mutex> lock(window_mutexes);
cv_empty.wait(lock, [this] { return empty_blocks.load() > 1; });
}
return group_w_idx.fetch_add(1);
}

size_t MemoryIdxPool::wait_next_full_group() {
size_t free_num = empty_blocks.fetch_add(1, std::memory_order_relaxed) + 1;
cv_empty.notify_all();

if (free_num >= MAX_GROUP_READ) {
std::unique_lock<std::mutex> lock(window_mutexes);
cv_filled.wait(lock, [this] { return empty_blocks.load() < MAX_GROUP_READ; });
}
return group_r_idx.fetch_add(1);
}

bool MemoryIdxPool::check_group() {
bool result = (group_w_idx.load() > group_r_idx.load()) ? true : false;
return result;
}
Loading