Merge remote-tracking branch 'origin/dramsim'

Author: gktrk

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "src/DRAMSim2"]
+	path = src/DRAMSim2
+	url = https://github.com/umd-memsys/DRAMSim2.git

Changelog.md

@@ -1,3 +1,8 @@
+# Version 2.0a
+- Added [DRAMSim2](https://github.com/umd-memsys/DRAMSim2) support
+- Flush all the CPU caches and DRAM models for every new simulation run
+- Fixed Issue #8: useless cleaning of local variables
+
 # Version 1.1a
 - Added 16550A UART support (thanks to Marc Gauthier)
 - Reworked the dram latency parameters to match the Sifive HiFive U540 Board

README.md

@@ -2,9 +2,9 @@
 
 [![MIT license](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/bucaps/marss-riscv/blob/master/src/MIT-LICENSE.txt) [![GitHub issues](https://img.shields.io/github/issues/Naereen/StrapDown.js.svg)](https://github.com/bucaps/marss-riscv/issues/)
 
-MARSS-RISCV (Micro-ARchitectural System Simulator - RISCV) is a open source, **cycle-accurate single core full-system (Linux) micro-architectural simulator** for the [RISC-V](https://riscv.org/specifications/) ISA built upon [TinyEMU emulator (https://bellard.org/tinyemu)](https://bellard.org/tinyemu) by Fabrice Bellard and uses its code for all the device emulation and configuration. It consists of detailed cycle accurate models of a modern RISC-V In-order and Out-of-order processor with branch prediction unit and a complete memory hierarchy. It is currently being developed and maintained by [CAPS](https://github.com/bucaps/) (Computer Architecture and Power Aware Systems Research Group) at the State University of New York at Binghamton. Being a true full system simulator, MARSS-RISCV can simulate all of the system in a cycle accurate fashion including OS code, libraries, interrupt handlers etc.
+MARSS-RISCV (Micro-ARchitectural System Simulator - RISCV) is a open source, **cycle-accurate single core full-system (Linux) micro-architectural simulator** for the [RISC-V](https://riscv.org/specifications/) ISA built upon [TinyEMU emulator (https://bellard.org/tinyemu)](https://bellard.org/tinyemu) by Fabrice Bellard and uses its code for all the device emulation and configuration. It consists of detailed cycle accurate models of a modern RISC-V In-order and Out-of-order processor with branch prediction unit and a complete memory hierarchy including TLBs, caches and DRAM. It comes integrated with [DRAMSim2](https://github.com/umd-memsys/DRAMSim2), a cycle accurate memory system simulator. It is currently being developed and maintained by [CAPS](https://github.com/bucaps/) (Computer Architecture and Power Aware Systems Research Group) at the State University of New York at Binghamton. Being a true full system simulator, MARSS-RISCV can simulate all of the system in a cycle accurate fashion including OS code, libraries, interrupt handlers etc.
 
-Currently, our simulator is in alpha status as we are validating the cycle accuracy using various development boards. The simulated in-order core is tested and operational, however, the simulated out-of-order core is in microarchitectural testing phase.
+Currently, our simulator is in alpha status as we are validating the cycle accuracy using various development boards. The simulated in-order core is tested and operational, however, the simulated out-of-order core is in micro-architectural testing phase.
 
 ## Table of contents
 - [Features](#features)
@@ -23,7 +23,7 @@ Currently, our simulator is in alpha status as we are validating the cycle accur
 - Fully configurable, cycle-accurate, in-order and out-of-order single-core RISC-V CPU
 - Multiple execution units with configurable latencies (execution units can be configured to be pipelined)
 - 2-level cache hierarchy with various allocation and miss handling policies
-- A simple DIMM based DRAM model that simulates row-buffer (open-page) hits
+- 2 DRAM memory models: Simple DIMM based basic DRAM model that simulates row-buffer (open-page) hits and [DRAMSim2](https://github.com/umd-memsys/DRAMSim2)
 - Bi-modal and 2-level adaptive (Gshare, Gselect, GAg, GAp, PAg, PAp) branch prediction support
 - Supports `RV32GC` and `RV64GC` (user level ISA version `2.2`, privileged architecture version `1.10`)
 - VirtIO console, network, block device, input and 9P filesystem
@@ -55,12 +55,12 @@ First, clone the simulator repository:
 
 ```console
 $ git clone https://github.com/bucaps/marss-riscv
-
 ```
 Then, `cd` into the simulator source directory:
 
 ```console
 $ cd marss-riscv/src/
+$ git submodule update --init --recursive
 ```
 Modify `CONFIG_XLEN` and `CONFIG_FLEN` options in the Makefile as required. Supported `XLEN` values are `32` and `64`. Supported `FLEN` values are `0`, `32` and `64`. 
 
@@ -88,12 +88,14 @@ $ xz -d -k -T 0 riscv32.img.xz
 When decompression finishes, launch the simulator with:
 
 ```console
-$ ../../marss-riscv riscvemu.cfg
+$ ../../marss-riscv -mem-model base riscvemu.cfg
 ```
 
 Simulation parameters can be configured using `riscvemu.cfg`, RISCVEMU JSON configuration file. 
 
-By default, the simulator will boot in "snapshot" mode, meaning it will **not** retain the file system changes after it is shut down. In order to persist the changes, pass `-rw` command line argument to the simulator. In that case, it may also be desirable to grow the userland image (has roughly 200MB of available free space by default). More information about how to grow it can be found [here](https://github.com/bucaps/marss-riscv-images#how-to-use).
+By default, the simulator will boot in "snapshot" mode, meaning it will **not** retain the file system changes after it is shut down. In order to persist the changes, pass `-rw` command line argument to the simulator. MARSS-RISCV comes with 2 DRAM memory models: Basic and DRAMSim2. To specify which memory model to use, run MARSS-RISCV with command line option `-mem-model` and specify either `base` or `dramsim2`. For DRAMSim2, the paths to `ini` and `system ini file` can be specified in `riscvemu.cfg` file.
+
+It may also be desirable to grow the userland image (has roughly 200MB of available free space by default). More information about how to grow it can be found [here](https://github.com/bucaps/marss-riscv-images#how-to-use).
 
 By default, guest boots in emulation mode. To start in simulation mode run with `-simstart` command line option.
 
@@ -162,7 +164,6 @@ cycle = 115  pc = 0x6aaaadfc  insn = 0xfa842503  lw a0,s0,-88  mode = PRV_U  sta
 * Support for return address stack
 * Support for branch prediction unit, speculative execution and age-ordered instruction issue logic in the out-of-order core
 * Cycle accuracy validation using various RISC-V development boards
-* Integrating DRAM simulator (like DRAMSim2) with MARSS-RISCV, to get accurate memory stats
 
 ## Technical notes
 
@@ -231,8 +232,11 @@ the JSON blk.txt file must be provided as disk image filename.
 ## Acknowledgment
 This work was supported in part by DARPA through an award from the SSITH program. We would like to thank Gokturk Yuksek, Ravi Theja Gollapudi and Kanad Ghose for assistance with the internal details of TinyEMU and the development of the MARSS-RISCV.
 
+For DRAMSim2, see [here](https://github.com/umd-memsys/DRAMSim2).
+
 ## License
 This project is licensed under the MIT License - see the
 src/MIT-LICENSE.txt file for details.
 
 The SLIRP library has its own license (two clause BSD license).
+DRAMSim2 has its own license (two clause BSD license).

src/DRAMSim2

@@ -56,10 +56,10 @@ EXE=
 endif
 CC=$(CROSS_PREFIX)gcc
 STRIP=$(CROSS_PREFIX)strip
-CFLAGS=-O3 -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -MMD
+CFLAGS=-O3 -g -Wall -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -MMD -I./DRAMSim2
 CFLAGS+=-D_GNU_SOURCE -DCONFIG_VERSION=\"$(shell cat VERSION)\"
 LDFLAGS=
-
+EXT_LIB+= libdramsim.so libdramsim_wrapper_cpp_lib.so libdramsim_wrapper_c_connector.so
 bindir=/usr/local/bin
 INSTALL=install
 
@@ -80,7 +80,7 @@ endif
 
 PROGS+=
 
-all: clean $(PROGS) 
+all: clean $(EXT_LIB) $(PROGS)
 
 stats-display:
 	$(CC) stats_display.c -o stats-display -lrt
@@ -122,11 +122,23 @@ EMU_LIBS+=-lrt -lpthread -lm
 
 RISCVEMU_OBJS:=$(EMU_OBJS) riscvemu.o riscv_machine.o softfp.o
 
+libdramsim_wrapper_c_connector.so:
+	$(CXX) -g -DRV$(CONFIG_XLEN) -DSIM_FLEN=$(CONFIG_FLEN) -fpic -shared -o libdramsim_wrapper_c_connector.so riscvsim/dramsim_wrapper_c_connector.cpp -I./DRAMSim2 -L. -ldramsim_wrapper_cpp_lib -Wl,-rpath=. -std=c++11
+
+libdramsim_wrapper_cpp_lib.so:
+	$(CXX) -g -DRV$(CONFIG_XLEN) -DSIM_FLEN=$(CONFIG_FLEN) -fpic -shared -o libdramsim_wrapper_cpp_lib.so riscvsim/dramsim_wrapper.cpp -I./DRAMSim2 -L./DRAMSim2 -ldramsim -Wl,-rpath=./DRAMSim2 -std=c++11
+
+libdramsim.so:
+	@echo "Building DRAMSim2 shared library:"
+	@cd DRAMSim2/ && $(MAKE) libdramsim.so
+	@cd ..
+	@echo "Building MARSS-RISCV Simulator:"
+
 riscvsim.o: $(RISCV_SIM_OBJS)
 	$(LD) -r -o $@ $^ $(SIM_LIBS)
 
 marss-riscv$(EXE): $(RISCVSIM_OBJ_FILE) riscv_cpu.o $(RISCVEMU_OBJS)
-	$(CC) $(LDFLAGS) -o $@ $^ $(EMU_LIBS)
+	$(CC) $(LDFLAGS) -o $@ $^ $(EMU_LIBS)  -L. -ldramsim_wrapper_c_connector -Wl,-rpath=.
 
 riscvemu.o: riscvemu.c
 	$(COMPILE_DEBUG)$(CC) $(CFLAGS) -DCONFIG_XLEN=\"$(CONFIG_XLEN)\" -DCONFIG_FLEN=\"$(CONFIG_FLEN)\" -c -o $@ $<
@@ -150,7 +162,9 @@ install: $(PROGS)
 	$(COMPILE_DEBUG)echo "CC $<"
 
 clean:
-	rm -f *.o *.d *~ $(PROGS) slirp/*.o slirp/*.d slirp/*~ riscvsim/*.o riscvsim/*.d riscvsim/*~ 
+	rm -f *.o *.d *.so *~ $(PROGS) slirp/*.o slirp/*.d slirp/*~ riscvsim/*.o riscvsim/*.d riscvsim/*~
+	cd DRAMSim2 && $(MAKE) clean
+	cd ..
 
 -include $(wildcard *.d)
 -include $(wildcard slirp/*.d)

src/Makefile

@@ -193,6 +193,19 @@ static void parse_stage_latency_str(int **dest, int max_stage_count, char *str)
     }
 }
 
+static unsigned next_high_power_of_2(unsigned n)
+{
+    n--;
+
+    n |= n >> 1;
+    n |= n >> 2;
+    n |= n >> 4;
+    n |= n >> 8;
+    n |= n >> 16;
+
+    return ++n;
+}
+
 static int virt_machine_parse_config(VirtMachineParams *p,
                                      char *config_file_str, int len)
 {
@@ -899,40 +912,82 @@ static int virt_machine_parse_config(VirtMachineParams *p,
        }
     }
 
-    tag_name = "dram_burst_size";
-    if (vm_get_int(cfg, tag_name, (int *)&p->sim_params.dram_burst_size) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %u\n", tag_name,
-              p->sim_params.dram_burst_size);
-    }
 
-    tag_name = "mem_bus_access_rtt_latency";
-    if (vm_get_int(cfg, tag_name, &p->sim_params.mem_bus_access_rtt_latency) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
-              p->sim_params.mem_bus_access_rtt_latency);
-    }
+    switch (p->sim_params.mem_model_type)
+    {
+        case MEM_MODEL_BASE:
+        {
+            tag_name = "mem_bus_access_rtt_latency";
+            if (vm_get_int(cfg, tag_name, &p->sim_params.mem_bus_access_rtt_latency) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
+                      p->sim_params.mem_bus_access_rtt_latency);
+            }
 
-    tag_name = "tCL";
-    if (vm_get_int(cfg, tag_name, &p->sim_params.tCL) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
-              p->sim_params.tCL);
-    }
+            tag_name = "tCL";
+            if (vm_get_int(cfg, tag_name, &p->sim_params.tCL) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
+                      p->sim_params.tCL);
+            }
 
-    tag_name = "tRCD";
-    if (vm_get_int(cfg, tag_name, &p->sim_params.tRCD) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
-              p->sim_params.tRCD);
-    }
+            tag_name = "tRCD";
+            if (vm_get_int(cfg, tag_name, &p->sim_params.tRCD) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
+                      p->sim_params.tRCD);
+            }
 
-    tag_name = "tRP";
-    if (vm_get_int(cfg, tag_name, &p->sim_params.tRP) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
-              p->sim_params.tRP);
-    }
+            tag_name = "tRP";
+            if (vm_get_int(cfg, tag_name, &p->sim_params.tRP) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
+                      p->sim_params.tRP);
+            }
 
-    tag_name = "row_buffer_write_latency";
-    if (vm_get_int(cfg, tag_name, &p->sim_params.row_buffer_write_latency) < 0) {
-      fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
-              p->sim_params.row_buffer_write_latency);
+            tag_name = "row_buffer_write_latency";
+            if (vm_get_int(cfg, tag_name, &p->sim_params.row_buffer_write_latency) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %d\n", tag_name,
+                      p->sim_params.row_buffer_write_latency);
+            }
+            tag_name = "dram_burst_size";
+            if (vm_get_int(cfg, tag_name, (int *)&p->sim_params.dram_burst_size) < 0) {
+              fprintf(stderr, "%s not found, selecting default value: %u\n", tag_name,
+                      p->sim_params.dram_burst_size);
+            }
+            break;
+        }
+        case MEM_MODEL_DRAMSIM:
+        {
+            tag_name = "dramsim_ini_file";
+            if (vm_get_str(cfg, tag_name, &str) < 0) {
+                fprintf(stderr, "%s not found, selecting default value: %s\n", tag_name,
+                    p->sim_params.dramsim_ini_file);
+            } else {
+                free(p->sim_params.dramsim_ini_file);
+                p->sim_params.dramsim_ini_file = strdup(str);
+            }
+
+            tag_name = "dramsim_system_ini_file";
+            if (vm_get_str(cfg, tag_name, &str) < 0) {
+                fprintf(stderr, "%s not found, selecting default value: %s\n", tag_name,
+                    p->sim_params.dramsim_system_ini_file);
+            } else {
+                free(p->sim_params.dramsim_system_ini_file);
+                p->sim_params.dramsim_system_ini_file = strdup(str);
+            }
+
+            tag_name = "dramsim_stats_dir";
+            if (vm_get_str(cfg, tag_name, &str) < 0) {
+                fprintf(stderr, "%s not found, selecting default value: %s\n", tag_name,
+                    p->sim_params.dramsim_stats_dir);
+            } else {
+                free(p->sim_params.dramsim_stats_dir);
+                p->sim_params.dramsim_stats_dir = strdup(str);
+            }
+            break;
+        }
+        default:
+        {
+            fprintf(stderr, "error: invalid memory model\n");
+            exit(1);
+        }
     }
 
     /**
@@ -945,8 +1000,18 @@ static int virt_machine_parse_config(VirtMachineParams *p,
         p->sim_params.enable_bpu = FALSE;
     }
 
-    /* Setup remaining params */
-    p->sim_params.guest_ram_size = p->ram_size;
+    /* Set RAM size to be used by simulated memory model */
+    /**
+     *
+     * NOTE: TinyEMU reserves 2GB of physical address space for its devices and
+     * internal implementation. Guest RAM size taken as user input is mapped
+     * starting at 2GB. Hence for simulated memory model, the total guest ram
+     * size becomes (2GB + guest RAM size) rounded to next highest power of 2.
+     * This guest_ram_size is passed to either the base memory model or
+     * DRAMSim2, depending on which is being used.
+     */
+    p->sim_params.guest_ram_size
+        = next_high_power_of_2(2048 + (p->ram_size >> 20));
 
     json_free(cfg);
     return 0;

src/machine.c

@@ -45,6 +45,7 @@
 #include "iomem.h"
 #include "riscv_cpu.h"
 #include "sim_params_stats.h"
+#include "riscvsim/dramsim_wrapper_c_connector.h"
 
 static no_inline int target_read_slow(RISCVCPUState *s, mem_uint_t *pval,
                                       target_ulong addr, int size_log2);
@@ -320,7 +321,12 @@ stop_system_simulation(RISCVCPUState *s, target_ulong pc, uint64_t icount)
         fprintf(stderr, "(marss-riscv): Time elapsed on host-machine %lu ms\n",
                 sim_time);
 
-        dump_simulation_stats(s);
+	if (s->simcpu->mmu->mem_controller->mem_model_type == MEM_MODEL_DRAMSIM)
+	{
+		dramsim_wrapper_print_stats();
+	}
+
+	dump_simulation_stats(s);
 
 #if defined(CONFIG_SIM_TRACE)
         fclose(s->sim_trace);

src/riscv_cpu.c

@@ -616,6 +616,7 @@ static struct option options[] = {
     { "no-accel", no_argument },
     { "simstart", no_argument },
     { "stats-display", no_argument },
+    { "mem-model", required_argument },
     { NULL },
 };
 
@@ -629,13 +630,14 @@ void help(void)
 #ifdef CONFIG_CPU_RISCV
            "-b [32|64]    set the integer register width in bits\n"
 #endif
-           "-m ram_size       set the RAM size in MB\n"
-           "-rw               allow write access to the disk image (default=snapshot)\n"
-           "-ctrlc            the C-c key stops the emulator instead of being sent to the\n"
-           "                  emulated software\n"
-           "-append cmdline   append cmdline to the kernel command line\n"
-           "-simstart         start (boot kernel) in simulation mode\n"
-           "-stats-display    dump simulation performance stats to a shared memory location, read by sim-display tool\n"
+           "-m ram_size                 set the RAM size in MB\n"
+           "-rw                         allow write access to the disk image (default=snapshot)\n"
+           "-ctrlc                      the C-c key stops the emulator instead of being sent to the\n"
+           "                            emulated software\n"
+           "-append cmdline             append cmdline to the kernel command line\n"
+           "-simstart                   start (boot kernel) in simulation mode\n"
+           "-stats-display              dump simulation performance stats to a shared memory location, read by sim-display tool\n"
+           "-mem-model [base|dramsim2]  type of simulated memory model\n"
            "\n"
            "Console keys:\n"
            "Press C-a x to exit the emulator, C-a h to get some help.\n");
@@ -699,6 +701,7 @@ int main(int argc, char **argv)
     VirtMachineParams p_s, *p = &p_s;
     int marss_start_in_sim = 0;
     int marss_stats_display = 0;
+    int marss_mem_model = MEM_MODEL_BASE;
 
     ram_size = -1;
     allow_ctrlc = FALSE;
@@ -734,6 +737,21 @@ int main(int argc, char **argv)
             case 7: /* stats-display */
                 marss_stats_display = 1;
                 break;
+            case 8: /* mem-model */
+                if (strcmp(optarg, "base") == 0)
+                {
+                    marss_mem_model = MEM_MODEL_BASE;
+                }
+                else if (strcmp(optarg, "dramsim2") == 0)
+                {
+                    marss_mem_model = MEM_MODEL_DRAMSIM;
+                }
+                else
+                {
+                    fprintf(stderr, "unknown mem-model type, see help\n");
+                    exit(1);
+                }
+                break;
             default:
                 fprintf(stderr, "unknown option index: %d\n", option_index);
                 exit(1);
@@ -772,6 +790,7 @@ int main(int argc, char **argv)
     path = argv[optind++];
 
     virt_machine_set_defaults(p);
+    p->sim_params.mem_model_type = marss_mem_model;
 #ifdef CONFIG_FS_NET
     fs_wget_init();
 #endif