Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Run tests

on:
[push, pull_request]

jobs:
test:
name: Run tests and collect coverage
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 2

- name: Set up Python
uses: actions/setup-python@v4

- name: Install dependencies
run: pip install pytest pytest-cov pytest-pep8

- name: Build Python module
run: python3 setup.py sdist

- name: Install Python module
run: pip3 install ./dist/pgdbpool-1.0.1.tar.gz

- name: Run tests
run: pytest --cov --cov-branch --cov-report=xml
32 changes: 32 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,37 @@
# Changelog

## Version 1.0.1

### Bug Fixes

- **Default Configuration Values**: Added default values for optional database configuration parameters
- `query_timeout`: Default 5000 milliseconds (5 seconds)
- `session_tmp_buffer`: Default 128 MB
- `ssl`: Default 'disable'
- `connect_timeout`: Default 10 seconds
- Fixes issue where example `01-logical-replication` would not work without explicit configuration

### Changes

- **Connection Retry**: Removed configurable connect retry sleep time
- Connection retry sleep time is now statically set to 1 second
- Simplifies configuration by removing `connection_retry_sleep` parameter

### Documentation

- **Configuration Documentation**: Updated configuration documentation to reflect correct units and types
- Corrected `query_timeout` unit from Seconds to Milliseconds
- Corrected `ssl` type from boolean to enum (disable|allow|prefer|require)
- Updated default values to match implementation
- Simplified multi-database configuration examples to show only required parameters

### CI/CD

- **GitHub Actions**: Added GitHub Actions CI workflow
- Runs tests on push and pull request events
- Includes pytest with coverage reporting
- Validates module build and installation

## Version 1.0 (Stable)

- Stable release tested and verified
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

## 1. Primary Scope

The **pgdbpool** Python module is a tiny **PostgreSQL Database Connection De-Multiplexer**, primarily designed for *Web- / Application Servers*.
The **pgdbpool** Python module is a tiny **PostgreSQL Database Connection De-Multiplexer**.

**Key Features:**
- **Multi-endpoint support**: Load balance across multiple PostgreSQL servers
Expand All @@ -17,12 +17,12 @@ The **pgdbpool** Python module is a tiny **PostgreSQL Database Connection De-Mul

```text
+----------------------+ +---------------------
| WebServer Service.py | -- Handler Con #1 ----> | PostgreSQL
| Server Service.py | -- Handler Con #1 ----> | PostgreSQL
| Request / Thread #1 | | Backend #1
+----------------------+ |
|
+----------------------+ |
| WebServer Service.py | -- Handler Con #2 ----> | PostgreSQL
| Server Service.py | -- Handler Con #2 ----> | PostgreSQL
| Request / Thread #2 | | Backend #2
+----------------------+ +---------------------
```
Expand Down
56 changes: 17 additions & 39 deletions doc/conf.py/source/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,9 @@ The database connection configuration supports both single and multiple database
'user': 'username',
'pass': 'userpass',
'ssl': False,
'connect_timeout': 30,
'connect_timeout': 10,
'connection_retry_sleep': 1,
'query_timeout': 120,
'query_timeout': 5000,
'session_tmp_buffer': 128
}
}
Expand All @@ -65,23 +65,13 @@ The database connection configuration supports both single and multiple database
'host': 'db1.example.com',
'name': 'dbname',
'user': 'username',
'pass': 'userpass',
'ssl': False,
'connect_timeout': 30,
'connection_retry_sleep': 1,
'query_timeout': 120,
'session_tmp_buffer': 128
'pass': 'userpass'
},
{
'host': 'db2.example.com',
'name': 'dbname',
'user': 'username',
'pass': 'userpass',
'ssl': False,
'connect_timeout': 30,
'connection_retry_sleep': 1,
'query_timeout': 120,
'session_tmp_buffer': 128
'pass': 'userpass'
}
]
}
Expand Down Expand Up @@ -143,33 +133,27 @@ The threading model can be configured to optimize for different deployment scena
-
- Database Auth Password
* - ssl
- bool
- enum (disable|allow|prefer|require)
-
- x
- False
- Use SSL / TLS
-
- disable
- SSL / TLS properties
* - connect_timeout
- int
- Seconds
- x
- 30
-
- 10
- Connect Timeout
* - connection_retry_sleep
- int
- Seconds
- x
- 1
- Sleep Between Connect Retry
* - query_timeout
- int
- Seconds
- x
- 120
- Milliseconds
-
- 5000
- Query Timeout
* - session_tmp_buffer
- int
- Kilobytes
- x
- Megabytes
-
- 128
- Session Buffer Memory

Expand Down Expand Up @@ -230,9 +214,9 @@ The following schema represents the internal Python structures. Some values (e.g
'user': 'dbuser',
'pass': 'dbpass',
'ssl': False,
'connect_timeout': 30,
'connect_timeout': 10,
'connection_retry_sleep': 1,
'query_timeout': 120,
'query_timeout': 5000,
'session_tmp_buffer': 128
},
'groups': {
Expand Down Expand Up @@ -288,18 +272,12 @@ Configure multiple database endpoints for automatic load balancing and failover:
'name': 'myapp',
'user': 'appuser',
'pass': 'securepassword',
'ssl': 'require',
'connect_timeout': 30,
'query_timeout': 120
},
{
'host': 'secondary-db.example.com',
'name': 'myapp',
'user': 'appuser',
'pass': 'securepassword',
'ssl': 'require',
'connect_timeout': 30,
'query_timeout': 120
}
],
'groups': {
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "pgdbpool"
version = "1.0"
version = "1.0.1"
authors = [
{ name="Claus Prüfer", email="pruefer@webcodex.de" },
]
Expand Down
16 changes: 15 additions & 1 deletion src/pool.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,12 @@ def _init_class(cls):
cls._dbiter_ref = cls._dbiter()

db_config = cls._config['db'][0]

if 'query_timeout' not in db_config:
db_config['query_timeout'] = 5000
if 'session_tmp_buffer' not in db_config:
db_config['session_tmp_buffer'] = 128

statement_timeout = 'statement_timeout={}'.format(db_config['query_timeout'])
temp_buffers = 'temp_buffers={}MB'.format(db_config['session_tmp_buffer'])

Expand Down Expand Up @@ -216,10 +222,12 @@ def get_connection_count(cls, connection):
(group, group_id) = connection
cls.logger.debug('connection get_connection_count() group_id:{}'.format(group_id))
connections = cls._config['groups'][group]['connections']

for (conn_ref, status) in connections:
cls.logger.debug('connection get_connection_count() conn_ref:{} status:{}'.format(conn_ref, status))
if status == 'occupied':
connection_count += 1

return connection_count

@classmethod
Expand All @@ -236,6 +244,7 @@ def set_connection_status(cls, connection, status):
connection_by_id = connections[group_id]
new_connection = (connection_by_id[0], status)
connections[group_id] = new_connection

cls.logger.debug('connection set_connection_status() group_id:{} status:{} con_ref:{}'.format(
group_id,
status,
Expand Down Expand Up @@ -271,6 +280,11 @@ def connect(cls, connection):

cls.logger.debug('connection connect() db_config:{}'.format(db_container))

if 'ssl' not in db_container:
db_container['ssl'] = 'disable'
if 'connect_timeout' not in db_container:
db_container['connect_timeout'] = 10

group_container['connections'][conn_id] = (
psycopg2.connect(
dbname = db_container['name'],
Expand Down Expand Up @@ -316,7 +330,7 @@ def reconnect(cls, connection):
return
except DBConnectionError as e:
cls.logger.debug('connection reconnect() exception:{}'.format(repr(e)))
time.sleep(cls._config['db']['connection_retry_sleep'])
time.sleep(1)


class Query(object):
Expand Down
Loading