Customizing the filesystem / partitions for N16R8

[johandevlabs]

I have a ESP32-S3 N16R8 (https://www.robotics.org.za/ESP32-S3-N16R8).

The rebuild-esp32s3-linux-wifi.sh scripts works flawlessly on my device, but it is configured to N8R8 (with 8MB flash total).

How do I reconfigure the build to make full use of the 16MB flash? Ideally, I would like to allocate an additional 8MB to the rootfs (and make it read/write) so I can place files in the /root folder. Is this possible?

~ # ls /dev/mtd*
/dev/mtd0       /dev/mtd2ro     /dev/mtd5       /dev/mtdblock3
/dev/mtd0ro     /dev/mtd3       /dev/mtd5ro     /dev/mtdblock4
/dev/mtd1       /dev/mtd3ro     /dev/mtdblock0  /dev/mtdblock5
/dev/mtd1ro     /dev/mtd4       /dev/mtdblock1
/dev/mtd2       /dev/mtd4ro     /dev/mtdblock2
~ # df -h
Filesystem                Size      Used Available Use% Mounted on
mtd:rootfs                4.2M      4.2M         0 100% /
mtd:etc                 448.0K    216.0K    232.0K  48% /etc

[jcmvbkbc]

I would like to allocate an additional 8MB to the rootfs (and make it read/write) so I can place files in the /root folder.
Is this possible?

Making it rw -- no, not without losing the XIP capability. But you could create a new partition, format it however you like
and mount it to /root or elsewhere. For that you'd need to start with the partition table in the bootloader, make your version based on the default and use it in the bootloader configuration. You'd also need to change the selected flash size in the config.
Then you can add that new partition to the /etc/fstab in the buildroot rootfs overlay here. And then you'd either need to build an image for that file system and flash it like the other partitions, or reconfigure the buildroot to include FLASH tools and do formatting from the running linux system.

[johandevlabs]

Thanks @jcmvbkbc, this makes sense.

For context, I am trying to include Micropython and Netcat (nc). First step for me was to expand the rootfs so I have space for the additional packages. I forked your esp-hosted fork and configured it for N16R8 https://github.com/johandevlabs/esp-hosted/tree/ipc-5.1.1_N16R8 ). It is working now! :) I will tinker more with adding a new partition later.

/etc # df -h
Filesystem                Size      Used Available Use% Mounted on
mtd:rootfs                6.5M      6.5M         0 100% /
mtd:etc                 448.0K    220.0K    228.0K  49% /etc

I have test nc and it works for POST requests. I have also tested micropython, unfortunately it does not work yet

/etc # micropython 
[ 1738.801573] micropython: page allocation failure: order:9, mode:0xcc0(GFP_KERNEL), nodemask=(null)
[ 1738.802777] CPU: 0 PID: 214 Comm: micropython Not tainted 6.10.0-rc2 #1
[ 1738.807069] Stack:
[ 1738.809199] > 00000100 00000000 3df69bb0 423be718
[ 1738.813860]   423be718 3d87d50b 3dee8920 3d8850a8
[ 1738.818599]   8217e67b 3df69c00 00000000 3dee8c78
[ 1738.823230]   3df69be0 00000004 00060900 00000000
[ 1738.827901]   8217eb7d 3df69c20 00000cc0 00000000
[ 1738.832585] > 3df69c3c 00000cc0 3df69c50 00000000
[ 1738.837267]   8217ebd9 3df69ca0 00000000 00000009
[ 1738.841946] > 00000000 00000000 00000000 00000000
[ 1738.846643]   3df69c80 3df69c60 0000000c 423be072
[ 1738.851331]   3df69c30 00000000 3d87da8c 00000cc0
[ 1738.856004]   00000cc0 00000000 00000000 00000005
[ 1738.860695]   00000005 00000000 0000001b 00000009
[ 1738.865387]   0000000c 3d87da8c 00000000 3df69c90
[ 1738.870067]   00000001 00000009 00000cc0 00000000
[ 1738.875082]   8217ecec 3df69d20 00000000 00000cc0
[ 1738.879523] > 00000001 00000000 00000000 00000000
[ 1738.884097] Call Trace:
[ 1738.886499]  [<42376e8d>] 0x42376e8d
[ 1738.890083]  [<4217e67b>] 0x4217e67b
[ 1738.893613]  [<4217eb7d>] 0x4217eb7d
[ 1738.897159]  [<4217ebd9>] 0x4217ebd9
[ 1738.900701]  [<4217ecec>] 0x4217ecec
[ 1738.904263]  [<4217ed65>] 0x4217ed65
[ 1738.907818]  [<4217ba6c>] 0x4217ba6c
[ 1738.911377]  [<42176823>] 0x42176823
[ 1738.914940]  [<4217bc5f>] 0x4217bc5f
[ 1738.918530]  [<4217bc7e>] 0x4217bc7e
[ 1738.922061]  [<42120770>] 0x42120770
[ 1738.925611]  [<421204fa>] 0x421204fa
[ 1739.045983] Mem-Info:
[ 1739.046688] active_anon:0 inactive_anon:0 isolated_anon:0
[ 1739.046688]  active_file:0 inactive_file:6 isolated_file:0
[ 1739.046688]  unevictable:24 dirty:0 writeback:0
[ 1739.046688]  slab_reclaimable:173 slab_unreclaimable:805
[ 1739.046688]  mapped:0 shmem:0 pagetables:0
[ 1739.046688]  sec_pagetables:0 bounce:0
[ 1739.046688]  kernel_misc_reclaimable:0
[ 1739.046688]  free:421 free_pcp:0 free_cma:0
[ 1739.135769] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:24kB unevictable:96kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:320kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
[ 1739.183967] Normal free:1680kB boost:0kB min:340kB low:424kB high:508kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:24kB unevictable:96kB writepending:0kB present:8192kB managed:7324kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 1739.222099] lowmem_reserve[]: 0 0
[ 1739.222938] Normal: 30*4kB (U) 31*8kB (U) 18*16kB (U) 8*32kB (U) 4*64kB (U) 2*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1680kB
[ 1739.238654] 32 total pagecache pages
[ 1739.239363] 2048 pages RAM
[ 1739.239482] 0 pages HighMem/MovableOnly
[ 1739.239613] 217 pages reserved
[ 1739.241311] nommu: Allocation of length 1052672 from process 214 (micropython) failed
[ 1739.273510] Mem-Info:
[ 1739.274179] active_anon:0 inactive_anon:0 isolated_anon:0
[ 1739.274179]  active_file:0 inactive_file:6 isolated_file:0
[ 1739.274179]  unevictable:24 dirty:0 writeback:0
[ 1739.274179]  slab_reclaimable:173 slab_unreclaimable:805
[ 1739.274179]  mapped:0 shmem:0 pagetables:0
[ 1739.274179]  sec_pagetables:0 bounce:0
[ 1739.274179]  kernel_misc_reclaimable:0
[ 1739.274179]  free:420 free_pcp:0 free_cma:0
[ 1739.353913] Node 0 active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:24kB unevictable:96kB isolated(anon):0kB isolated(file):0kB mapped:0kB dirty:0kB writeback:0kB shmem:0kB writeback_tmp:0kB kernel_stack:320kB pagetables:0kB sec_pagetables:0kB all_unreclaimable? no
[ 1739.383634] Normal free:1680kB boost:0kB min:340kB low:424kB high:508kB reserved_highatomic:0KB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:24kB unevictable:96kB writepending:0kB present:8192kB managed:7324kB mlocked:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[ 1739.408307] lowmem_reserve[]: 0 0
[ 1739.410645] Normal: 30*4kB (U) 31*8kB (U) 18*16kB (U) 8*32kB (U) 4*64kB (U) 2*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1680kB
[ 1739.431429] 32 total pagecache pages
[ 1739.432124] 2048 pages RAM
[ 1739.432241] 0 pages HighMem/MovableOnly
[ 1739.433514] 217 pages reserved
Segmentation fault

I build the image with a fork of your buildroot fork (https://github.com/johandevlabs/buildroot/tree/xtensa-2024.05-fdpic_mpy_nc )

[jcmvbkbc]

I have also tested micropython, unfortunately it does not work yet
...
[ 1739.241311] nommu: Allocation of length 1052672 from process 214 (micropython) failed

Yeah, there's no way it would succeed on a system with 8M PSRAM.

[johandevlabs]

Thanks for the hint @jcmvbkbc . I got it to work now (with option -X heapsize=100000)

/etc # micropython -X heapsize=100000
MicroPython v1.22.0 on 2024-09-20; linux [GCC 14.1.0] version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> print("thanks for hint Max! Memory is key")
thanks for hint Max! Memory is key
>>> for i in range(3):
...     print(i)
... 
0
1
2
>>> 
/etc # free -m
              total        used        free      shared  buff/cache   available
Mem:              7           5           2           0           1           1
/etc # free -k
              total        used        free      shared  buff/cache   available
Mem:           7324        4792        1588           0         944        1488

It is amazing that one can run Linux with WiFi and everything on such a small device!

[Mintaka]

Hi everybody.

Thanks for sharing your work with ESP32-S3.
I have big plans with programmable smart watch build on ESP32-S3 N16R8
https://lilygo.cc/products/t-watch-s3
But, after many days of probing, I see, I am still on the beginning.

One branch of my experiments was tying to follow your successful steps, build and run Linux on that piece of hardware.
But I failed.

run jcmvbkbc
https://github.com/jcmvbkbc/esp32-linux-build/blob/master/rebuild-esp32s3-linux-wifi.sh
(not under root user)

works fine, but after flashing, the only response was

linux ptr = 0x42120000
vectors ptr = 0x4037c000

as mentioned in #9

====

In hope that cite johandevlabs: "everything works out of the box. Nice work!" #9 (comment)

I install fresh distribution of Debian 12 to virtualbox.
Linux desp32 6.1.0-28-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1 (2024-11-22) x86_64 GNU/Linux

Then install
sudo apt install autoconf automake bash bc bison build-essential cmake flex gawk git gperf help2man libncurses-dev libtool libtool-bin libusb-1.0-0 python3 python3-venv rsync texinfo unzip wget

and run
https://github.com/johandevlabs/esp32s3-linux-custom-build/blob/master/rebuild-esp32s3-linux-wifi-mpy-nc.sh

Sadly, here stop me this problem in pcre2 10.43 Building

>>> pcre2 10.43 Building
GIT_DIR=. PATH="/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/host/bin:/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/host/sbin:/home/u/mywork/lilygo/esp32s3-linux-custom-build/autoconf-2.71/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games"  /usr/bin/make -j5  -C /home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43/
make[1]: Entering directory '/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43'
rm -f src/pcre2_chartables.c
ln -s /home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43/src/pcre2_chartables.c.dist /home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43/src/pcre2_chartables.c
/usr/bin/make  all-am
make[2]: Entering directory '/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43'
  CC       src/pcre2grep-pcre2grep.o
  CC       src/libpcre2_8_la-pcre2_auto_possess.lo
  CC       src/libpcre2_8_la-pcre2_chkdint.lo
  CC       src/libpcre2_8_la-pcre2_compile.lo
  CC       src/libpcre2_8_la-pcre2_config.lo
src/pcre2grep.c: In function 'pcre2grep_callout':
src/pcre2grep.c:2506:7: error: implicit declaration of function 'fork' [-Wimplicit-function-declaration]
 2506 | pid = fork();
      |       ^~~~
make[2]: *** [Makefile:2833: src/pcre2grep-pcre2grep.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43'
make[1]: *** [Makefile:1544: all] Error 2
make[1]: Leaving directory '/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43'
make: *** [package/pkg-generic.mk:283: /home/u/mywork/lilygo/esp32s3-linux-custom-build/build/build-buildroot-esp32s3/build/pcre2-10.43/.stamp_built] Error 2
make: Leaving directory '/home/u/mywork/lilygo/esp32s3-linux-custom-build/build/buildroot'
+ '[' -f build-buildroot-esp32s3/images/xipImage -a -f build-buildroot-esp32s3/images/rootfs.cramfs -a -f build-buildroot-esp32s3/images/etc.jffs2 ']'
+ exit 1

Larger output is here
https://codefile.io/f/56H7Ppu936

Can you please advise me?

[Mintaka]

Sorry, "works fine" is not he right status :)

More precisely, the build and compilation were completed and the flashing was done without errors.

But the running was not successful.

[jcmvbkbc]

works fine, but after flashing, the only response was

linux ptr = 0x42120000
vectors ptr = 0x4037c000

I couldn't quickly figure out which interface is used for programming and where you expect the console output when kernel boot process starts, but if it's not the UART0 then you wouldn't see any output unless the kernel is reconfigured to use the correct device for console. See part of this page that describes moving serial console to the USB serial. You can put the text earlycon=esp32s3acm,mmio32,0x60038000 console=ttyGS3 debug rw root=mtd:rootfs no_hash_pointers into the file build/build-buildroot-esp32s3_devkit_c1_8m/target/etc/cmdline and run keep_toolchain=y keep_buildroot=y keep_rootfs=y keep_bootloader=y nice ./rebuild-esp32s3-linux-wifi.sh to rebuild and reflash the etc filesystem.

[jcmvbkbc]

run
https://github.com/johandevlabs/esp32s3-linux-custom-build/blob/master/rebuild-esp32s3-linux-wifi-mpy-nc.sh

Sadly, here stop me this problem in pcre2 10.43 Building
...
src/pcre2grep.c:2506:7: error: implicit declaration of function 'fork' [-Wimplicit-function-declaration]
...

This package obviously needs to be adapted to be built for noMMU. However since it's produced by the script from some other repository, please report it there.

[Mintaka]

Thanks for advice.

My device https://lilygo.cc/products/t-watch-s3 have only one micro-usb port which is detect as:
Bus 001 Device 036: ID 303a:1001 Espressif USB JTAG/serial debug unit
and connect as:
/dev/ttyACM0

I'll try
earlycon=esp32s3acm,mmio32,0x60038000 console= ...

And sorry for writing error about script
https://github.com/johandevlabs/esp32s3-linux-custom-build/blob/master/rebuild-esp32s3-linux-wifi-mpy-nc.sh
here, issues section is not present there.

[Mintaka]

Move my next conversation to more appropriate topic #9