![]() ![]() When we come across the Series record, after creation of the series, we look for the series reference in the above map and if any memory-mapped chunks exist, we attach that list to this series.We then continue with the WAL replay as described in Part 2 but with few modification: If the chunk was in the file 00093 and the series ref starts at byte offset 1234 in the file, then the reference of that chunk would be (93 in the memory. The first 4 bytes tell the file number in which the chunk exists, and the last 4 bytes tell the offset in the file where the chunk starts (i.e. Reading these chunks įor every chunk, the Head block stores the mint and maxt of that chunk along with a reference in the memory to access it. len is the number of bytes that follow from here and data are the actual bytes of the compressed chunk.ĬRC32 is the checksum of the above content of the chunk used to check the integrity of the data. encoding is the encoding used to compress the chunks. The mint and maxt are the minimum and maximum timestamp seen in the samples of the chunk. The series ref is the same series reference that we talked about in Part 2, it is the series id used to access the series in the memory. These chunks stay in its own directory called chunks_head and have a file sequence similar to WAL (except it starts with 1). The immutability is the most important factor here else rewriting compressed chunks would have been too inefficient for every sample. This flushed chunk is the memory-mapped chunk from disk. Recapping from Part 1, when a chunk is full, we cut a new chunk and the older chunks become immutable and can only be read from (the yellow block below).Īnd instead of storing it in memory, we flush it to disk and store a reference to access it later. ![]() I have also given a KubeCon talk on this which explains this at a little higher level. We will be diving deeper into how this is designed in Prometheus in this blog post.Īs this is a part of the Prometheus TSDB blog series that I am writing, you are recommended to read the Part 1 to know where these memory mapped chunks fit into TSDB (or the Head block) and Part 2 to understand the WAL replay. This helps in reducing the memory footprint of the Head block and also helps speed up the WAL replay that we discussed in Part 2. In the Part 1 of the TSDB blog series I mentioned that once a chunk is "full", it is flushed to the disk and memory mapped. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |