FS performance notes

src/SUMMARY.md

@@ -8,7 +8,7 @@
         - [Actor System]()
         - [Security Features](development/design/security.md)
         - [Microkernel](development/design/kernel.md)
-        - [GUI]()
+        - [GUI](development/design/gui.md)
         - [Filesystem](development/design/filesystem.md)
     - [Configuring a Build Environment](development/environment.md)
     - [Development Workflow](development/workflow.md)

src/development/README.md

@@ -34,7 +34,7 @@ The source for this site, and our [website](https://mercury.the-system.eu.org) i
 All `crates`/`libraries` are in a `no-std` environment. This means we only have access to the [libcore](https://doc.rust-lang.org/core/) functionality.
 However, we will be using the `alloc` crate to access the heap, and`collections` to have access to data structures like `Vec`.
 
-We should, however, have basic support for [async](https://ferrous-systems.com/blog/stable-async-on-embedded/) and [threading]() in `core::`.
+We should, however, have basic support for [async](https://ferrous-systems.com/blog/embedded-concurrency-patterns/) and [threading]() in `core::`.
 
 ## Learning
 Before jumping in, I highly recommend learning some stuff abotu **Rust** and embedded development with it.

src/development/design/README.md

@@ -64,10 +64,10 @@ erDiagram
 ### Startup Flow
 ```mermaid
 flowchart TD
-    [[Bootloader]] --> kern(Kernel)
-    kern --> disk(Read Disk) -->
+    boot[Bootloader] --> kern(Kernel)
+    kern --> disk(Read Disk) --> ind(Index Filesystem) -->
     parse(Parse Configuration) --> run(Run Startup Programs)
     parse -.-> sh([Interactive Shell])
-    kern --> mem(Map Memory) -.-> parse
+    kern --> mem(Map Memory) -.-> ind
     run ==> actor([Create Actors])
 ```

src/development/design/filesystem.md

@@ -15,37 +15,54 @@ They can save at any time, save immediately, or just save on a *shutdown* signal
 Therefore, the "filesystem" code will just be a library that's simple a low-level interface for the `kernel` to use.
 *Actors* will simply make requests to save.
 
+## Performance
+I believe that this format should be fairly fast, but only implementation and testing will tell for sure.
+1. Minimal data needs to read in - bit offsets can be used, and only fixed-size metadata must be known
+2. `serde` is fairly optimized for deserialization/serialization
+3. `HighwayHash` is a very fast and well-optimized hashing algorithm
+4. Async and multithreading will allow for concurrent access, and splitting of resource-intensive tasks across threads
+5. `hashbrown` is quite high-performance
 
 ## Filesystem Layout
+
+| Name | Size | Header |
+|------|------|--------|
+| Boot Sector | `128 B` | `None` |
+| Kernel Sector | `4096 KB` | `None` |
+| Index Sector | `4096 KB` | `None` |
+| Config Sector | `u64` | `PartitionHeader` |
+| User Sector(s) | `u64` | `PartitionHeader` |
+
 ### Partition
 A virtual section of the disk.
-It's identified simply by numerical order.
+Additionally, it has a **UUID** generated via [lolid](https://lib.rs/crates/lolid) to enable identifying a specific partition.
+
 ```rust
-const BOOT_SIZE: u64; // How large the BOOT partition will be
-const LABEL_SIZE: u64; // Number of characters that can be used in the partition label
+const LABEL_SIZE: u16 = 128; // Example number of characters that can be used in the partition label
 
 struct PartitionHeader {
-    boot: bool, // Boot flag
     label: [char; LABEL_SIZE], // Human-readable label. Not UTF-8 though :/
     num_chunks: u64, // Chunks in this partition
+    uuid: Uuid,4096
 }
 ```
 
 ### Chunk
 Small pieces that each partition is split into.
-Contains fixed-length metadata (checksum, extension flag, uuid) at the beginning, and then arbitrary data afterwards.
-If the saved data exceeds past a single chunk, the `extends` flag is set.
-
-Additionally, it has a **UUID** generated via [lolid](https://lib.rs/crates/lolid) to enable identifying a specific chunk.
+Contains fixed-length metadata (checksum, encryption flag, modification date, etc.) at the beginning, and then arbitrary data afterwards.
 
 ```rust
-const CHUNK_SIZE: u64; // Example static chunk size
+const CHUNK_SIZE: u64 = 4096; // Example static chunk size
+
+struct ChunkHeader {
+    checksum: u64,
+    encrypted: bool,
+    modified: u64, // Timestamp of last modified
+    uuid: Uuid,
+}
 
 struct Chunk {
-    checksum: u64,
-    extends: bool,
-    encrypted: bool,
-    uuid: Uuid,
+    header: ChunkHeader,
     data: [u8; CHUNK_SIZE],
 }
 ```
@@ -53,21 +70,21 @@ This struct is then encoded into bytes and written to the disk. Drivers for the
 It *should* be possible to do autodetection, and maybe for *Actors* to specify which disk/partition they want to be saved to.
 
 Compression of the data should also be possible, due to `bincode` supporting [flate2](https://lib.rs/crates/flate2) compression.
-Similarely **AES** encryption can be used, and this allows for only specific chunks to be encrypted.[^encryption]
+Similarly **AES** encryption can be used, and this allows for only specific chunks to be encrypted.[^encryption]
 
 ### Reading
-On boot, we start executing code from the beginning of the disk (the boot partition, although that's meaningless at this point).
-The `kernel` then reads in bytes from the first partition *(as the **BOOT** partition is fixed-size, we know when this starts)* into memory, serializing it into a `PartitionHeader` struct via [bincode](https://lib.rs/crates/bincode).
+On boot, we start executing code from the **Boot Sector**. This contains the assembly instructions, which then jump to the `kernel` code in the **Kernel Sector**.
+The `kernel` then reads in bytes from the first partition *(as the sectors are fixed-size, we know when this starts)* into memory, serializing it into a `PartitionHeader` struct via [bincode](https://lib.rs/crates/bincode).
 
 From here, as we have a fixed `CHUNK_SIZE`, and know how many chunks are in our first partition, we can read from any chunk on any partition now.
-On startup, an *Actor* can request to read data from the disk. If it has the right [capabilities](/development/design/actor.md#ocap), we find the chunk it's looking for[^find_chunk], parse the data (using `bincode` again), and send it back.
+On startup, an *Actor* can request to read data from the disk. If it has the right [capabilities](/development/design/actor.md#ocap), we find the chunk it's looking for from the index, parse the data (using `bincode` again), and send it back.
 
 Also, we are able to verify data. Before passing off the data, we re-hash it using [HighwayHash](https://lib.rs/crates/highway) to see if it matches.
 If it does, we simply pass it along like normal. If not, we refuse, and send an error [message](/development/design/actor.md#messages).
 
 ### Writing
 Writing uses a similar process. An *Actor* can request to write data. If it has proper capabilties, we serialize the data, allocate a free chunk[^free_chunk], and write to it.
-We *hash* the data first to generate a checksum, and set proper metadata if the data extends past the `CHUNK_SIZE`.
+We *hash* the data first to generate a checksum, and set proper metadata.
 
 ### Permissions
 Again, whether actors can:
@@ -77,9 +94,44 @@ Again, whether actors can:
 
 will be determined via [capabilities](/development/design/actor.md#ocap)
 
+### Indexing
+Created in-memory on startup, modified directly whenever the filesystem is modified.
+It's saved in the *Index Sector* (which is at a known offset & size), allowing it to be read in easily on boot.
+It again simply uses `bincode` and compression.
+
+While the index is not necessarily a fixed size, we read until we have enough data from the fixed sector size.
+
+```rust
+use hashbrown::HashMap;
+
+let mut index = HashMap::new(); // Create the Uuid storage index
+let mut free_index = HashMap::new(); // Create the freespace index
+
+struct Location {
+    partition: Uuid, // Partition identified via Uuid
+    chunks: Vec<u64>, // Which chunk(s) in the partition it is
+}
+
+let new_data = (Uuid::new(), b"data"); // Test data w/ an actor Uuid & bytes
+let new_data_location = Location {
+    partition: Uuid::new(),
+    chunks: vec![5, 8], // 5th & 8th chunk in that partition
+};
+
+index.insert(&new_data.0, &new_data_location); // Insert a new entry mapping a data Uuid to a location
+
+let uuid_location = index.get(&new_data.0).unwrap(); // Get the location of a Uuid
+```
+
+This then allows the index to be searched easily to find the data location of a specific `Uuid`.
+Whenever an actor makes a request to save data to it's `Uuid` location, this can be easily found.
+It also allows us to tell if an actor *hasn't* been saved yet, allowing us to know whether we need to allocate new space for writing, or if there's actually something to read.
+
 ### To-Do
 - Snapshots
 - Isolation
+- Journaling
+- Resizing
 
 ## Executable Format
 Programs written in userspace will need to follow a specific format.
@@ -109,6 +161,4 @@ struct PackedExecutable {
 
 [^encryption]: Specific details to be figured out later
 
-[^find_chunk]: Currently via magic. I have no idea how to do this other than a simple search. Maybe generate an index, or use a **UUID**?
-
-[^free_chunk]: Again, no idea how.
+[^free_chunk]: Need to figure out how to efficiently do this. **XFS** seems to just keep another index of free chunks. It also uses a **B+Tree** rather than a hashmap - to look into.

src/development/design/gui.md

@@ -1 +1,38 @@
 # GUI
+Eventually, programs will be able to use the `photon` library to have access to a graphics API.
+This will initialize various [actors](/development/design/actor.md) to represent parts of the UI.
+
+## Drawing
+When a **GUI** element wants to update, it first sends a [message](/development/design/actor.md#messages) to the `kernel`.
+The `kernel` then calculates the overlaying of each window, writes each window to it's own buffer, then updates the screen buffer with ones that have changed, which is then drawn to the screen.
+This ensures that only necessary parts are re-rendered, and the rendering can be done asynchronously/threaded.
+
+The `photon` library will not only provide a high-level API for applications to use, but also lower-level drawing methods for the `kernel` to use.
+These may include line, rectangle, triangle, and circle drawing methods, as well as being able to render text.
+
+### Flow
+```mermaid
+flowchart LR
+    app(Application) --> kern(Kernel)
+    kern --> buf([Buffer])
+    kern --> app
+    buf --> dis((Display))
+```
+
+## Styling
+Styling of GUI elements is done via a global configuration.
+The `kernel` parses this information, and uses it to actually style the widgets provided to it.
+
+Widgets created by the program/library contain no styling data - only information such as text, size, callbacks, etc.
+The `kernel` does all the display work.
+
+## Design
+```mermaid
+erDiagram
+    WINDOW ||--|{ SECTION: holds
+    WINDOW ||--|| TITLE: has
+    SECTION ||--|| TITLE: has
+    SECTION ||--o{ SECTION: holds 
+    SECTION ||--o{ CANVAS: holds
+    SECTION ||--o{ TEXTBOX: holds
+```