Feature Request: Any chance we could get system prompt caching in this fork?

[fantasyz]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

This fork has some caching features that caught my eyes:
https://github.com/fewtarius/CachyLLama
I particularly desire the system prompt caching because it affect every new session in agentic coding.
Hard cache the system prompt section and never let go as it always need to be used when using coding agents.

Motivation

I saw the reprocessing of the first 20K context happening very often. (Qwen-code CLI initial prompt with some setup is 20K large.) I noticed that even if I run a small task and start another new session, it still do the 20K re-processing for about 15s~20s.
Since the system prompt for each session is basically fixed. Imo, it is worth it for caching it even if just in RAM and don't persist across restart. Multiple of 15s adds up and lots of time can be saved.

Possible Implementation

Can reference to the CachyLLama project for inspiration.

[Anbeeld]

Yes, that's a very interesting feature, I'll look into it.