RAM usage setup #168

arita37 · 2021-12-31T14:25:39Z

arita37
Dec 31, 2021

Hi,

Few questions :

How to limit Maximum RAM usage for Qdrant ?
(ie storing the Index)
and use instead the ssd disk.
Or some reference of ram storage cost
of 1 million , 768-vectors ?
Is there any reference
For a given size of data:
10 million data points, vect 512
20k/requests ?
Is there a way to download directly binaries
on linux ?

Thanks

generall · 2021-12-31T15:39:22Z

generall
Dec 31, 2021
Maintainer

Hi @arita37! Thanks for your interest in the project!

How to limit Maximum RAM usage for Qdrant ?

There is, currently, no direct configuration for this in the service. You can try to limit RAM usage using docker.
Use --memory="1024mb" flag to limit total allowed memory per container. This method, however, won't prevent out-of-memory errors. It will only make them happen earlier.
To make the service use disk instead of RAM, you might want to adjust memmap_threshold in the config file: https://qdrant.tech/documentation/install/#configuration

It will force the service to switch vector storage to the mmaped files, starting from the given number of vectors per segment. The downside is slower retrieval speed - check out our benchmark for details: https://github.com/qdrant/benchmark/tree/master/memory_benchmark

10 million data points, vect 512

Looks like data of this size would require distributed deployment, please follow our roadmap https://github.com/qdrant/qdrant/pull/158/files for updates on this feature

Is there a way to download directly binaries on linux ?

We haven't published binaries yet, but it should be possible to extract them from docker container.
If you could describe your use-case and why it might be useful to have binaries instead of docker, we would start publishing them with the next release.

PS: I am going to convert this issue into a discussion

0 replies

arita37 · 2021-12-31T16:15:37Z

arita37
Dec 31, 2021
Author

Sure,

Does it work with
15 million data points on 32 Gb RAM, 8 cores, single machine ?
it seems 0.4.2 docker release does NOT allow this flag :

performance:
# Number of parallel threads used for search operations. If 0 - auto selection.
max_search_threads: 6

https://github.com/qdrant/qdrant/blob/v0.4.2/config/production.yaml

Is your docker regularly in line with latest config ?
(had to add manully debug:false in the config file).

why it starts 32 dockers even I put number of thread 6 ?

2021-12-31T16:35:40Z INFO actix_server::builder] Starting 32 workers
Should it runs only on full machine ?

Sometimes, have a shared machine which is itself a docker, so prefer running the binaries itself.
instead of docker in docker...

8 replies

arita37 Dec 31, 2021
Author

Ok, thx.
On same server/client have those speed

Is it due to network ?
why an int becomes a list of int (??)

Creating a new table
#### Insert data  ##############################
(3, 4) float32 [[0.48924324 0.5875858  0.46373758 0.30846044]
 [0.02925851 0.55592984 0.6670837  0.83807534]
 [0.23421568 0.7609976  0.20684677 0.5723214 ]]
[{'ran_id': 0, 'genre_id': 3}, {'ran_id': 1, 'genre_id': 3}, {'ran_id': 2, 'genre_id': 3}]

**1it [00:05,  5.03s/it]**

#### Search data  ##############################
[{'genre_id': [3], 'ran_id': [80]}, {'genre_id': [3], 'ran_id': [62]}, {'genre_id': [3], 'ran_id': [50]}, {'genre_id': [3], 'ran_id': [79]}, {'genre_id': [3], 'ran_id': [57]}]

generall Dec 31, 2021
Maintainer

currently, that is an expected behavior. All payload values are stored as arrays

arita37 Jan 2, 2022
Author

Sure.
Just wondering if there is any config reason for such slowness:
( 4 thread for search),
size of db is 100 vectors,...

#### Insert data  ##############################
(3, 4) float32 [[0.9120517  0.9262601  0.14656663 0.5392592 ]
 [0.51315576 0.461954   0.9804225  0.8682675 ]
 [0.96242225 0.6412142  0.7924407  0.09958279]]
[{'ran_id': 0, 'genre_id': 2}, {'ran_id': 1, 'genre_id': 2}, {'ran_id': 2, 'genre_id': 2}]
1it [00:05,  5.02s/it]

time 5.027352809906006

#### Search data  ##############################
[{'genre_id': [3], 'ran_id': [80]}, {'genre_id': [3], 'ran_id': [49]}, {'genre_id': [3], 'ran_id': [79]}, {'genre_id': [3], 'ran_id': [62]}, {'genre_id': [3], 'ran_id': [57]}]

time 10.038408517837524

generall Jan 2, 2022
Maintainer

Sure.
Just wondering if there is any config reason for such slowness:
( 4 thread for search),
size of db is 100 vectors,...

it doesn't look right 🤔
what is the timing of internal timer in the API?

Could you please describe a minimal setup of how to reproduce that?

arita37 Jan 2, 2022
Author

Sure, thanks
This is the config, and python code :

https://gist.github.com/arita37/ffc3c83dd9e111093c98a4987f5c0f82

Suppose this is related to the config/setup...

arita37 · 2022-01-02T12:32:23Z

arita37
Jan 2, 2022
Author

Or maybe the network part….
but, think if the connection is persistent after 1st connection (?(

1 reply

generall Jan 2, 2022
Maintainer

I can see from your log, from the service side response was pretty quick:

[2022-01-02T04:29:50Z INFO  actix_web::middleware::logger] 100.XXXX:23619 "POST /collections/test/points/search HTTP/1.1" 200 319 "-" "-" 0.000647

0.000647 - is time in seconds

But 10 seconds seems too long even for establishing 1st time connection, are you using some sort of satellite connection?

but, think if the connection is persistent after 1st connection (?(

Default client uses this configuration for keep-alive:

KEEPALIVE_EXPIRY = 5.0 # seconds
Limits(max_connections=100, max_keepalive_connections=20)

But you can alter them by passing **kwargs to QdrantClient.

E.g.

# Disable keep-alive
client = QdrantClient(limits=httpx.Limits(max_connections=None, max_keepalive_connections=0))

arita37 · 2022-01-02T13:09:32Z

arita37
Jan 2, 2022
Author

Thx, gd news, It seems coming from underlying network…. Will try with persistence of 60sec. Also, how to spill out the the logs on disk ? (through docker command?)

…

On Jan 2, 2022, at 21:52, Andrey Vasnetsov ***@***.***> wrote: I can see from your log, from the service side response was pretty quick: [2022-01-02T04:29:50Z INFO actix_web::middleware::logger] 100.XXXX:23619 "POST /collections/test/points/search HTTP/1.1" 200 319 "-" "-" 0.000647 0.000647 - is time in seconds But 10 seconds seems too long even for establishing 1st time connection, are you using some sort of satellite connection? but, think if the connection is persistent after 1st connection (?( Default client uses this configuration for keep-alive: KEEPALIVE_EXPIRY = 5.0 # seconds Limits(max_connections=100, max_keepalive_connections=20) But you can alter them by passing **kwargs to QdrantClient. E.g. # Disable keep-alive client = QdrantClient(limits=httpx.Limits(max_connections=None, max_keepalive_connections=0)) — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

1 reply

generall Jan 2, 2022
Maintainer

Do you mean something like this?

docker logs %container_id% > log.txt

arita37 · 2022-01-02T23:46:42Z

arita37
Jan 2, 2022
Author

Sure. Was rather thinking of the real time logs of qdrant (now its jusr printing on screen). Not sure, maybe some —logs mylog_dirhost/ on qdrant CLI thx

…

On Jan 2, 2022, at 22:41, Andrey Vasnetsov ***@***.***> wrote: Do you mean something like this? docker logs %container_id% > log.txt — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you were mentioned.

0 replies

arita37 · 2022-01-03T02:57:51Z

arita37
Jan 3, 2022
Author

Using url directly, have those numbers,
a bit faster.

#### Create Collection CURL ##############################
 curl -X POST 'http://XXX.XXX.XXX:6333/collections' \
>     -H 'Content-Type: application/json' \
>     --data-raw '{
>         "create_collection": {
>             "name": "test2",
>             "vector_size": 4,
>             "distance": "Dot"
>         }
>     }'

{"result":true,"status":"ok","time":1.6756186739999999}
time:    6.793895483016968   ### python client side measured  6sec !!!



### Same speed than inside docker.
2022-01-03T02:54:09Z INFO  wal] Wal { path: "/data/collections/test2/wal", segment-count: 1, entries: [0, 0)  }: opened
[2022-01-03T02:54:09Z INFO  wal::segment] Segment { path: "/data/collections/test2/wal/open-2", entries: 0, space: (8/33554432) }: created
[2022-01-03T02:54:09Z INFO  actix_web::middleware::logger] XXX:28536 "POST /collections HTTP/1.1" 200 48 "-" "curl/7.65.3" 1.631060


### but ping is very fast... (ie client and server on same host...)
PING XXXX (XXX) 56(84) bytes of data.
64 bytes from XXX: icmp_seq=1 ttl=64 time=0.061 ms
64 bytes from XXX: icmp_seq=2 ttl=64 time=0.048 ms
64 bytes from XXXX: icmp_seq=3 ttl=64 time=0.070 ms

Trying to figure how to workadound the network part.
Maybe, access without HTTP part through the command line ?

1 reply

arita37 Jan 4, 2022
Author

Thx for various comments.
To close on this topic:
network slowness was solved with some proxy tweak (ie outside of qdrant part).
For people who want to have fast acces one can do as follow:

 Docker  -it  clientdocker --link qdrant01
 
 it would link on same localhost qdrant and server, reducing any latency (ie for some batching process...).

Faiee50 · 2024-07-18T07:01:25Z

Faiee50
Jul 18, 2024

Hi @generall, is there a method in Qdrant to limit memory usage without encountering out-of-vocabulary (OOV) errors? Additionally, is it possible to keep the most recently used collections in RAM, even when using memmap to store values in each segment?

2 replies

timvisee Jul 18, 2024
Maintainer

What are 'out-of-vocabulary (OOV)'?

If you're using memory mapping, the most recently accessed data is automatically kept in memory. Your kernel takes care of that.

With regards to limiting memory, generally speaking this can be done by setting memory limits on the Qdrant Docker container.

Faiee50 Jul 18, 2024

Thanks @timvisee ,.. Sorry it is out of memory error.if you look at this old comment by @generall , it is saying that limiting RAM in docker will give OOM error, if the ram is reached to its mentioned limit.
https://github.com/orgs/qdrant/discussions/168#discussioncomment-1891743

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Qdrant

RAM usage setup #168

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 7 comments 13 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Qdrant

RAM usage setup #168

arita37 Dec 31, 2021

Replies: 7 comments · 13 replies

generall Dec 31, 2021 Maintainer

arita37 Dec 31, 2021 Author

arita37 Dec 31, 2021 Author

generall Dec 31, 2021 Maintainer

arita37 Jan 2, 2022 Author

generall Jan 2, 2022 Maintainer

arita37 Jan 2, 2022 Author

arita37 Jan 2, 2022 Author

generall Jan 2, 2022 Maintainer

arita37 Jan 2, 2022 Author

generall Jan 2, 2022 Maintainer

arita37 Jan 2, 2022 Author

arita37 Jan 3, 2022 Author

arita37 Jan 4, 2022 Author

Faiee50 Jul 18, 2024

timvisee Jul 18, 2024 Maintainer

Faiee50 Jul 18, 2024

arita37
Dec 31, 2021

Replies: 7 comments 13 replies

generall
Dec 31, 2021
Maintainer

arita37
Dec 31, 2021
Author

arita37 Dec 31, 2021
Author

generall Dec 31, 2021
Maintainer

arita37 Jan 2, 2022
Author

generall Jan 2, 2022
Maintainer

arita37 Jan 2, 2022
Author

arita37
Jan 2, 2022
Author

generall Jan 2, 2022
Maintainer

arita37
Jan 2, 2022
Author

generall Jan 2, 2022
Maintainer

arita37
Jan 2, 2022
Author

arita37
Jan 3, 2022
Author

arita37 Jan 4, 2022
Author

Faiee50
Jul 18, 2024

timvisee Jul 18, 2024
Maintainer