We will design, develop, and deploy a distributed service to explore what Go can do. We’ll develop and deploy the service in layers: from the bare essentials of storage handling, to the networking of a client and server, to distributing server instances, deployment, and testing. I divided this book into four parts that parallel those layers.
Chapter 1: Test Your API
Chapter 3: ???
Each segment comprises a store file and an index file. The segment’s store file is where we store the record data; we continually append records to this file. The segment’s index file is where we index each record in the store file.
Index files are small enough that we can memory-map them and make operations on the file as fast as operating on in-memory data.
Memory-mapping isn’t necessarily size-constrained, so I’m not sure I follow. Data is still making it to disk eventually, so isn’t this simply trading explicit IO latency for implicit/hidden IO latency? I’m not seeing the benefit of using
mmap here yet.
- Record: the data stored in our log.
- Store: the file we store records in.
- Index: the file we store index entries in.
- Segment: the abstraction that ties a store and an index together
- Log: the abstraction that ties all the segments together.
The reason we resize them now is that, once they’re memory-mapped, we can’t resize them, so it’s now or never.
Why does the index store offsets? Isn’t an entry’s relative position in the index sufficient to determine its offset?
Chapter 4: ???
Internal packages are magical packages in Go that can only be imported by nearby code. For example, you can import code in /a/b/c/internal/d/e/f by code rooted by /a/b/c, but not code rooted by /a/b/g.
gRPC optionally supports status codes (not related to HTTP status codes