Der Bericht Andrei Borodin berichtet, wie sie die Skalierungserfahrung von PgBouncer bei der Entwicklung des Compounds Pullers Odyssey bei der Einführung in die Produktion berücksichtigt haben . Darüber hinaus werden wir diskutieren, welche Funktionen des Abziehers wir in neuen Versionen sehen möchten: Es ist wichtig, dass wir nicht nur unsere Bedürfnisse abdecken, sondern auch die Odyssey- Benutzergemeinschaft entwickeln .
Video:
Hallo! Ich heiĂźe Andrey.
Bei Yandex entwickle ich Open Source-Datenbanken. Und heute haben wir ein Thema ĂĽber Verbindungspooler-Verbindungen.
Wenn Sie wissen, wie man Verbindungspooler auf Russisch anruft, sagen Sie es mir. Ich möchte wirklich einen guten Fachbegriff finden, der in der Fachliteratur gut etabliert sein sollte.
, connection pooler . - , , , Postgres . ( HighLoad++ 2019) Postgres. , , , , , , . , .
Postgres. . . , Postgres.
– primary , ( ), , .
– Postgres, Postgres connection pooler. Connection pooler fencing .
connection pooler?
Postgres . , – , Postgres. , .
, Postgres , procArray. . procArray , . , . , .
3 :
- .
- .
- , . . .
, pooler . PostgreSQL Professional . , . . application-side pool proxy pool.
Application-side pool – . : .
, , .
, , -. client side pooling . – 10 000 . , .
proxy poolers, poolers, . poolers. poolers + . Pgpool Crunchy-Proxy.
, , . , poolers pooling, . . , .
, PgBouncer, transaction pooling, . . .
– . .
, , . loopback - .
, application_name_add_host. Bouncer IP- application_name. application_name .
, – , – , . – application_name, , .
, Bouncer pool, . . , .
? , C++ - , , . 20 000 , . .
, , Bouncer, , . . pool.
Postgres, . . .
, . PgBouncer , . : , , , , , - . . , , .
, .
, Bouncer . . , , pooler.
, PgBouncer’.
https://lwn.net/Articles/542629/
Bouncer.
, TCP- Bouncers. TCP- round-robin’ .
, . . , Bouncer, idle- Bouncer’.
, 3 Bouncers 100 %. Bouncers. ?
TLS. . Postgres TLS TLS, , , TLS handshake .
, . primary , – . . . - primary , -. TLS.
TLS handshake Bouncer, . - . retry exponential backoff, .
16 PgBouncer, 16 100 %.
PgBouncer. , Bouncer. Bouncers TCP handshake, Bouncers pooling, , .
. 18 Bouncers . . , DevOps , , .
, open source, Bouncer . , , PgBouncers . pull request .
https://www.postgresql.org/docs/current/libpq-cancel.html
https://github.com/pgbouncer/pgbouncer/pull/79
. Postgres , - . TCP-reset, . . . Bouncer? . . , , Bouncer , , .
upstream Bouncer’.
, connection pooler, , , , , .
. TLS-.
, Machinarium, . libpq, , : « . IO, , IO ». . . " N, N ", " SYNC, ". , . , . , , (green thread-). , , .
, TCP accept round-robin' workers TPC-.
. cache-friendly.
, , TCP-stack.
, , Odyssey CANCEL ROLLBACK , . . , Odyssey , , .
. application_name_add_host. , , .
.. managed PostgreSQL connection pooler, , . . , , . Bouncer .
.
, . , , , . , connection pooler Odyssey, .
Odyssey PgBouncer. , . - , pull request, issue GitHub, . PgBouncer .
, , error forwarding. , . , , , .
, 100%- PgBouncer. , Bouncer, .
Odyssey.
https://github.com/yandex/odyssey/pull/66
, «Pause /Resume». . Postgres, connection pooler, pg_upgrade, resume. , . . , . ( )
https://github.com/yandex/odyssey/pull/73 —
, PgBouncer, SCRAM Authentication, , .. – .
Odyssey, .
Odyssey, . Kiwi – Postgres’ . . . proto 3 Postgres – , , . Kiwi.
Machinarium – . Machinarium . , , 15 .
Odyssey. , coroutines. accept TCP- workers.
worker . crone- , pool.
Odyssey Postgres. install-check Bouncer Odyssey, div. , Bouncer Odyssey.
, , . Odyssey.
, - : Postgres + Odyssey, PgBouncer + Odyssey, Odyssey + Odyssey , , , Odyssey - , - , .
Odyssey production. , , . , . . , . , production , PostgreSQL Professional , memory leak. , . .
, connection pooler TLS- TLS-. .
Bouncer Odyssey pcache, pcache , Odyssey . , . , 20 000 .
Pluggable Authentication Method – lunux’ . PgBouncer , PAM PgBouncer, PAM.
. . ?
, , PAM- PAM-, PAM- PAM-. , . , .
, , accept . worker pool, TLS handshake.
, 20 000 , . libpq . 3 .
, , retry.
, PgBouncer , throttling TCP-, accept.
, , , , . , handshake , . - , .
Roadmap
Odyssey? ?
2019 .
roadmap Odyssey :
- SCRAM PAM authentication.
- forward standby.
- online-restart.
- .
roadmap , . . , .
forward read-only queries to standby? , . failover switchover. - - . , -, .
, Postgres, 10- session_attrs. , : . , , session_attrs.
, . - , . , , Odyssey , .
Odyssey primary. , , , , . .
, open source. , , 2,5 PgBouncer. Odyssey.
prepared statement. prepared statement . -, SQL-, «prepared». SQL- SQL Bouncer. overkill, , . . . SQL-.
prepared statement proto3. , , prepared statement, . , - prepared statements. , - .
, - , , prepared statements , , . . prepared statement.
, , prepared statement, . , , , , , , , .
, . , PgBouncer. . – : - , - – . .
, , , , .
– 1.0 ( 1.1). , Odyssey 1.0rc, . . release candidate. , , , memory leak.
1.0? Odyssey . , 1 000 000 , , , 1.0.
, 1.0 SCRAM. , production , SCRAM, . , , .
pull request. , Bouncer. . , - , .
, . !
application_name, , transaction pooling Odyssey?
Odyssey Bouncer?
Odyssey. Bouncer .
.
, ?
, . , application_name. , . . , .
, , ! ! , Odyssey . . multi data-source , Odyssey , . . slave, failover .
, , , . storages. . , , failover, pg_recovery. , . - ? . . , . . , C, .
, adoption . , . . , .
. , . , , , . , , , . , , , , , . . . , , .
, . pcache , , real cache , , , .
- , , , , . . .
. , . , , , 10 . , .
, .
. . , , . , .
Nginx hat eine solche Option slowly startim Cluster für den Server. Und er erhöht allmählich die Last.
Ja, groĂźartige Idee, wir werden es versuchen, wenn wir dazu kommen.