pageserver: improve IO concurrency for in-memory layers

[VladLazar]

Our story for serving writes and reads to the in-memory layer at the same time is really bad.
If we need to read from the in-memory layer while under a heavy write workload, get_page and ingest
performance degrade significantly due to the locking schemes used.

The in-memory layer is composed of: the backing file, in flight flush buffer and mutable tail.
Note that the read path should never read from the mutable tail and that the backing file is append only.

We can improve the locking schemes to allow: concurrent IO on the file itself and lock-less reads to the flush buffer. #12151 proposes a solution for the former.