Question
This Content is from Stack Overflow. Question asked by Tiger
String: 70036 01:11:34 ERROR GetItemFromSessionStore => StackExchange.Redis.RedisTimeoutException: The timeout was reached before the message could be written to the output buffer, and it was not sent
(5000ms, inst=37, qs=37, in=29426, active=EVAL), inst: 37, qs: 37, in: 0, serverEndpoint: Unspecified/***redis.redis.cache.windows.net:6380,
mgr: 10 of 10 available, clientName: ******,
IOCP: (Busy=3,Free=997,Min=4,Max=1000),
WORKER: (Busy=5582,Free=27185,Min=5680,Max=32767), v: 2.0.519.65453 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)
at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in C:projectsstackexchange-redissrcStackExchange.RedisConnectionMultiplexer.cs:line 2175
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor1 processor, ServerEndPoint server) in C:projectsstackexchange-redissrcStackExchange.RedisRedisBase.cs:line 54
at StackExchange.Redis.RedisDatabase.ScriptEvaluate(String script, RedisKey[] keys, RedisValue[] values, CommandFlags flags) in C:projectsstackexchange-redissrcStackExchange.RedisRedisDatabase.cs:line 1134
We are getting a lot of these errors when we get a surge of requests. We did have CPU and very high thread count when the timeout exceptions were logged.
Based on the documentation,
qsQueue-Awaiting-Response : {int}There are x operations currently awaiting replies from redis server.
inInbound-Bytes : {long}there are x bytes waiting to be read from the input stream from redis
Redis server seems to be handling the load well. So can someone explain what the error means?
Solution
Yes, likely thread starvation. You can set the minimum threshold so it doesn’t run out of threads. The basic theory is the OS has to wait, like 500 ms, before it can spin up another thread after it reaches the minimum threshold.
ThreadPool.SetMinThreads(minWorker, minIoc)
https://docs.microsoft.com/en-us/dotnet/api/system.threading.threadpool.setminthreads?view=net-6.0
Answered by beautifulcoder
This Question and Answer are collected from stackoverflow and tested by JTuto community, is licensed under the terms of CC BY-SA 4.0.