Multi-layer caching architecture practice in VODK project

这个网站是做什么的

tv.odkkk.com is a small site I made. Simply put, it is a video aggregation portal. It does not store movie sources itself, but instead collects content from several resource sites on the Internet, so that users no longer have to jump back and forth to find movies.

It mainly does three things:

• Search the entire network at once: Enter a film title, go to multiple resource sites in the background and ask again, merge the results, remove duplicates, sort them and return. Users see one clean list, rather than having titles pasted between five or six sites.
• Complete information: The data returned by the resource station is usually very rough, often only the title and playback address. I will take these to Douban to check the ratings, introductions, posters, and supplement the information about Bangumi to make every result decent.
• Remember where you saw: Playback progress, search history, and favorites all follow the account and will not be lost when changing pages.

It doesn’t sound complicated, but when you actually do it, you find that the problem is not the function itself, but the fact that the data source is too unreliable. The resource site frequently times out and is blocked. Douban gets 403 when browsing too much, but the homepage recommendations cannot always display last week’s content. If you go directly to the source to pull data every time you click on it, the experience will be very bad - sometimes it opens in seconds, sometimes it takes a long time to spin, and sometimes it reports an error directly.

Therefore, the key to whether this website can be used is not what the front end looks like, but whether the upstream “wind” can be hidden in the data layer, so that users will always feel that it is smooth. This is the problem that the following cache architecture will solve.

Core idea: The closer the data is to the user, the better

The entire data layer borrows the idea of ​​​​CPU cache - from the fastest to the slowest, layer by layer:

请求 → 边缘缓存 → 热数据缓存 → 持久存储 → 上游源站

Each level only asks questions when it misses, and returns directly when it hits. This not only maximizes the response, but also minimizes the pressure on the upstream. Let’s talk about it layer by layer.

First layer: edge cache

The outermost is the CDN edge node. For the same request, as long as someone has asked it recently, the edge node will directly return the last response to you as it is, without reaching my server at all. This is like stocking up on best-selling products in the canteen downstairs, without having to go to the main warehouse every time.

Here’s a little trick: let expired responses be returned first, and quietly updated in the background. What the user gets is the “old data from the previous second”, but it will be refreshed the next second - it always starts in seconds, so there is no need to wait for the source to be returned.

Second layer: hot data cache

The edge cache misses and the request reaches the application layer. There is a memory-level fast cache here, which is dedicated to data that has been frequently accessed recently.

The smartest thing about it is that it will judge its own popularity: the more a piece of data is accessed, the longer it will be retained. The cold data that just comes in is only kept for one minute, but if it starts to be clicked repeatedly, it will be automatically extended to a few minutes, ten minutes, or half an hour. In turn, content that no one reads will quickly expire and take up space.

The result is: the more popular the content, the more permanent it will be, and the unpopular content will naturally be eliminated. There is no need for people to guess “which ones should be kept”, the visitors will vote for themselves.

There is another detail: when writing data, I will save a “backup copy”, and the validity period is five times that of the original. If the original copy happens to be expired and is being returned to the source, the user can at least get the old data in full instead of staring at the loading circle.

The third layer: persistent storage

Hot cache data will not really disappear after it expires - it will be “warmed up” from a layer of persistent storage. This layer is saved for a long time, which is equivalent to an offline snapshot: even if the upstream site is completely down today, I still have the data captured last time to top it. At most, the user will see slightly older content instead of an error page.

Here I did two small things to avoid meaningless tossing:

First, it will only be overwritten if the content has really changed. Before each write, a “fingerprint” is calculated and compared. If the content does not change, only the access time and count are updated, without rewriting the data itself. In this way, those popular movies that are accessed repeatedly but the content has not changed will not be useless written in the database every day.

二是该新的数据得让它新。持久层是永久保留的，但有些东西不能一直喂旧数据——比如首页推荐，老显示一周前的榜单就没意思了。所以给每类数据设了个保鲜期，比如首页推荐一天必须刷新一次，搜索结果可以存得久一点。过了保鲜期，就算持久层有，也强制重新去源头拉一份。

The fourth layer: upstream origin station

If you really get to this point, it means that the first three levels have all missed. This is the time to really knock on the door of the resource station. The new data pulled back will be stuffed into the hot cache and persistence layer at the same time - and respond to the user before it is finished writing. The next time the same request is made, it will be hit directly from the cache.

Write the library out of the way

Writing data to the persistence layer is the most likely to slow down the response. So I made the writing “post-asynchronous”: first return the data to the user, and then slowly drop it into the database in the background. Even if the database writing fails, this access will not be affected. At most, the source will be returned once more next time.

If we separate “whether the user feels it is fast” and “whether the data is stored securely”, this trade-off is worth it - if the database is slower, the user will be indifferent, but if the response is half a second slower, the user will immediately become annoyed.

Effect

After running this architecture, the two most intuitive points are:

One is fast. The homepage opens basically in seconds, and searches basically take less than a hundred milliseconds. The vast majority of requests never hit the upstream.

The second is stability. A certain resource site has a problem today, and the user has no idea - because it is either in the hot cache or in the persistence layer, but the data is slightly older. When it recovers, it will naturally be updated next time it is crawled.

Write at the end

After completing this project, my view on “caching” has changed a lot. I used to think that caching was just “storing the results faster next time”, but now I think it’s more like digesting the upstream uncertainty for users - speed is just a by-product, stability is the real problem to solve.

A good caching strategy is not to store all data permanently, but to allow hot data to be permanent, cold data to be restored, and expired data to be refreshed.