Boost the performance of the inference by donhardman · Pull Request #146 · manticoresoftware/columnar

donhardman · 2026-03-26T13:56:00Z

No description provided.

… some cases

Related issues: - manticoresoftware/manticoresearch#3771 - manticoresoftware/manticoresearch#3869 The EmbedLib struct layout changed which is a breaking change in the embeddings library. Additional changes: - Added git commit ID and timestamp to the build process for consistent versioning across libraries. - Moved API key validation to CREATE TABLE. - Improved error handling for remote model requests, including detailed HTTP error reporting. - Updated tests to reflect changes in API key validation and model initialization.

- api_timeout=0 now means unlimited instead of 10s default - Jina, OpenAI, Voyage models accept None for no timeout - Update FFI test with api_url parameter

- Apply consistent multi-line formatting to error enum variants - Simplify timeout duration mapping using Option::map - Remove redundant match expressions in model constructors

- Integrate candle-onnx for ONNX model inference - Enable local ONNX model loading and execution - Add support for local embeddings using ONNX models

- Update candle to manticoresoftware fork - Switch candle dependencies to git revision

- Add protoc tool installation to embedding CI workflow - Install before main build in Linux job - Install conditionally for non-Linux distribution builds

- Add CUDA and Metal kernel support via candle-kernels and candle-metal-kernels - Integrate accelerate-src and intel-mkl-src for optimized CPU backends - Add ug, ug-cuda, ug-metal for unified GPU compute abstraction - Update candle-core, candle-nn, candle-transformers with new backend deps - Add hardware acceleration feature flags - Implement batched inference for BERT and ONNX with BATCH_SIZE 8 - Add predict_chunks and predict_batched helpers for tokenization and reassembly - Switch BERT/ONNX predict to batched path; keep sequential for T5/Causal/Quantized - Use attention-mask-aware mean pooling for accurate embeddings - Add ONNX vs Safetensors performance benchmark test

- Replaced std::iter::repeat().take() with std::iter::repeat_n() in BertEmbeddingModel and OnnxEmbeddingModel - repeat_n is more efficient as it avoids intermediate iterator overhead - Removed unused candle_onnx import

github-actions · 2026-03-30T11:57:08Z

Linux debug test results

8 files 8 suites 14m 17s ⏱️
517 tests 490 ✅ 27 💤 0 ❌
531 runs 504 ✅ 27 💤 0 ❌

Results for commit ebc6b5f.

♻️ This comment has been updated with latest results.

github-actions · 2026-03-30T12:02:55Z

Linux release test results

8 files 8 suites 7m 27s ⏱️
517 tests 497 ✅ 20 💤 0 ❌
531 runs 511 ✅ 20 💤 0 ❌

Results for commit ebc6b5f.

♻️ This comment has been updated with latest results.

- Upgrade actions/checkout and actions/cache from v3 to v4 - Split cargo cache into registry, target, and ML model caches - Add restore-keys fallback for master branch cache hits - Add Docker build caching with .cargo-cache volume mount - Add cleanup step to remove incremental build artifacts - Remove trailing whitespace in checkout step

- Remove separate win_bundle preparation job from test.yml - Move bundle download logic into win_test_template.yml - Use simple cache key instead of branch-specific cross-OS archive - Inline download when cache misses instead of pre-populating

github-actions · 2026-03-30T18:30:17Z

Windows test results

5 files 5 suites 17m 25s ⏱️
495 tests 475 ✅ 20 💤 0 ❌
503 runs 483 ✅ 20 💤 0 ❌

Results for commit ebc6b5f.

♻️ This comment has been updated with latest results.

github-actions · 2026-04-02T07:48:09Z

clt

❌ CLT tests in test/clt-tests/mcl/
✅ OK: 22
❌ Failed: 1
⏳ Duration: 250s
👉 Check Action Results for commit d2a7e8d

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/mcl/auto-embeddings-edge-cases.rec

test/clt-tests/mcl/auto-embeddings-edge-cases.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title) VALUES (2, 'Title without content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (4, '', 'Non-empty content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (5, '', '')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_unicode (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (1, '🔥 Machine Learning is awesome! 🚀💻')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (2, '机器学习和人工智能的未来')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (3, 'التعلم الآلي والذكاء الاصطناعي')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (4, 'English текст 日本語 mixed المختلط')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (5, '@#$%^&*()_+-=[]{}|;:<>?,./~\`')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_unicode WHERE KNN(vec, 2, '机器学习 🚀')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (100, 'Text with SQL: DROP TABLE test; SELECT * FROM users;')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT content_text FROM test_unicode WHERE id=100"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (101, 'Line 1
Line 2
	Tabbed line
Line 4')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (102, '<html><body><h1>Title</h1><script>alert(1)</script></body></html>')"; echo $?
––– output –––
OK
––– input –––
HUGE_TEXT=$(python3 -c "print('neural network ' * 1000)")
mysql -h0 -P9306 -e "CREATE TABLE test_huge (
    id BIGINT,
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_huge (id, content_text) VALUES (1, '$HUGE_TEXT')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, knn_dist() FROM test_huge WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_huge WHERE KNN(vec, 1, 'neural network')"
––– output –––
OK
––– input –––
for i in 1 2 3; do
    mysql -h0 -P9306 -e "INSERT INTO test_concurrent (id, title) VALUES ($((1000+i)), 'Concurrent insert $i')" 2>&1 &
done
wait
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_concurrent WHERE id >= 1001 AND id <= 1003"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_auto_dims (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_auto_dims"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_rapid (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
for i in $(seq 1 20); do
    mysql -h0 -P9306 -e "REPLACE INTO test_rapid (id, content_text) VALUES (1, 'Iteration $i')" 2>/dev/null
done
mysql -h0 -P9306 -e "SELECT content_text FROM test_rapid WHERE id=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_row (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty_row (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_empty_row WHERE KNN(vec, 1, 'test query')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_row (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_col (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_row (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_col (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_row WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_col WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_columnar (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='columnar'" --load="INSERT INTO test_scale_columnar (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_columnar"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_columnar;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_columnar ORDER BY id ASC LIMIT 100" > /tmp/test_scale_columnar.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_columnar WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_rowwise (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='rowwise'" --load="INSERT INTO test_scale_rowwise (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_rowwise"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_rowwise;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_rowwise ORDER BY id ASC LIMIT 100" > /tmp/test_scale_rowwise.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_rowwise WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_batch_perf (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s)
mysql -h0 -P9306 -e "INSERT INTO test_batch_perf (id, content) VALUES
(1, 'First text'), (2, 'Second text'), (3, 'Third text'),
(4, 'Fourth text'), (5, 'Fifth text')"
END_TIME=$(date +%s)
echo "Batch insert time: $((END_TIME - START_TIME)) seconds"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_row (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_col (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_row (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Row-wise 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_col (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Columnar 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_search (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'";
mysql -h0 -P9306 -e "SELECT id FROM test_empty_search WHERE KNN(vec, 5, 'search query')"; echo $?
––– output –––
OK

- Remove multi-chunk embedding averaging and get_mean_vector - Remove chunk_input_tokens utility - Add truncate_tokens to match Typesense behavior - Streamline prediction logic for local models - Replace chunking tests with truncation tests - BREAKING CHANGE: Removed public chunking utilities

github-actions · 2026-04-02T09:57:38Z

clt

❌ CLT tests in test/clt-tests/mcl/
✅ OK: 22
❌ Failed: 1
⏳ Duration: 232s
👉 Check Action Results for commit a45713b

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/mcl/auto-embeddings-edge-cases.rec

test/clt-tests/mcl/auto-embeddings-edge-cases.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title) VALUES (2, 'Title without content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (4, '', 'Non-empty content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (5, '', '')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_unicode (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (1, '🔥 Machine Learning is awesome! 🚀💻')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (2, '机器学习和人工智能的未来')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (3, 'التعلم الآلي والذكاء الاصطناعي')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (4, 'English текст 日本語 mixed المختلط')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (5, '@#$%^&*()_+-=[]{}|;:<>?,./~\`')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_unicode WHERE KNN(vec, 2, '机器学习 🚀')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (100, 'Text with SQL: DROP TABLE test; SELECT * FROM users;')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT content_text FROM test_unicode WHERE id=100"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (101, 'Line 1
Line 2
	Tabbed line
Line 4')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (102, '<html><body><h1>Title</h1><script>alert(1)</script></body></html>')"; echo $?
––– output –––
OK
––– input –––
HUGE_TEXT=$(python3 -c "print('neural network ' * 1000)")
mysql -h0 -P9306 -e "CREATE TABLE test_huge (
    id BIGINT,
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_huge (id, content_text) VALUES (1, '$HUGE_TEXT')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, knn_dist() FROM test_huge WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_huge WHERE KNN(vec, 1, 'neural network')"
––– output –––
OK
––– input –––
for i in 1 2 3; do
    mysql -h0 -P9306 -e "INSERT INTO test_concurrent (id, title) VALUES ($((1000+i)), 'Concurrent insert $i')" 2>&1 &
done
wait
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_concurrent WHERE id >= 1001 AND id <= 1003"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_auto_dims (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_auto_dims"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_rapid (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
for i in $(seq 1 20); do
    mysql -h0 -P9306 -e "REPLACE INTO test_rapid (id, content_text) VALUES (1, 'Iteration $i')" 2>/dev/null
done
mysql -h0 -P9306 -e "SELECT content_text FROM test_rapid WHERE id=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_row (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty_row (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_empty_row WHERE KNN(vec, 1, 'test query')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_row (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_col (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_row (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_col (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_row WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_col WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_columnar (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='columnar'" --load="INSERT INTO test_scale_columnar (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_columnar"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_columnar;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_columnar ORDER BY id ASC LIMIT 100" > /tmp/test_scale_columnar.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_columnar WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_rowwise (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='rowwise'" --load="INSERT INTO test_scale_rowwise (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_rowwise"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_rowwise;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_rowwise ORDER BY id ASC LIMIT 100" > /tmp/test_scale_rowwise.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_rowwise WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_batch_perf (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s)
mysql -h0 -P9306 -e "INSERT INTO test_batch_perf (id, content) VALUES
(1, 'First text'), (2, 'Second text'), (3, 'Third text'),
(4, 'Fourth text'), (5, 'Fifth text')"
END_TIME=$(date +%s)
echo "Batch insert time: $((END_TIME - START_TIME)) seconds"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_row (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_col (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_row (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Row-wise 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_col (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Columnar 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_search (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'";
mysql -h0 -P9306 -e "SELECT id FROM test_empty_search WHERE KNN(vec, 5, 'search query')"; echo $?
––– output –––
OK

- Replace candle-onnx with ort for model inference - Add ndarray and ort as core dependencies - Remove candle-onnx, candle-core, and prost-related dependencies - Implement manual mean pooling for 3D tensor outputs - Configure ORT session with Level 3 optimization - Enable automatic binary downloading for ort - Remove candle tensor and device dependencies

- Bump rust-min-libc images to version 1.94.1 - Remove manual oneMKL installation for amd64 builds

- Switch ureq and ort TLS backends to rustls - Remove OpenSSL and native security framework crates - Disable default features for ort dependency - Downgrade windows-sys versions for compatibility - Remove unused dependencies including der and tracing

github-actions · 2026-04-02T15:48:54Z

clt

❌ CLT tests in test/clt-tests/mcl/
✅ OK: 22
❌ Failed: 1
⏳ Duration: 233s
👉 Check Action Results for commit fe422a8

Failed tests:

🔧 Edit failed tests in UI:

Edit test/clt-tests/mcl/auto-embeddings-edge-cases.rec

test/clt-tests/mcl/auto-embeddings-edge-cases.rec

––– input –––
rm -f /var/log/manticore/searchd.log; stdbuf -oL searchd --stopwait > /dev/null; stdbuf -oL searchd ${SEARCHD_ARGS:-} > /dev/null
––– output –––
OK
––– input –––
if timeout 10 grep -qm1 'accepting connections' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Accepting connections!'; else echo 'Timeout or failed!'; fi
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title) VALUES (2, 'Title without content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (4, '', 'Non-empty content')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty (id, title, content) VALUES (5, '', '')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_unicode (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (1, '🔥 Machine Learning is awesome! 🚀💻')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (2, '机器学习和人工智能的未来')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (3, 'التعلم الآلي والذكاء الاصطناعي')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (4, 'English текст 日本語 mixed المختلط')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (5, '@#$%^&*()_+-=[]{}|;:<>?,./~\`')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_unicode WHERE KNN(vec, 2, '机器学习 🚀')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (100, 'Text with SQL: DROP TABLE test; SELECT * FROM users;')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT content_text FROM test_unicode WHERE id=100"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (101, 'Line 1
Line 2
	Tabbed line
Line 4')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_unicode (id, content_text) VALUES (102, '<html><body><h1>Title</h1><script>alert(1)</script></body></html>')"; echo $?
––– output –––
OK
––– input –––
HUGE_TEXT=$(python3 -c "print('neural network ' * 1000)")
mysql -h0 -P9306 -e "CREATE TABLE test_huge (
    id BIGINT,
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_huge (id, content_text) VALUES (1, '$HUGE_TEXT')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, knn_dist() FROM test_huge WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_huge WHERE KNN(vec, 1, 'neural network')"
––– output –––
OK
––– input –––
for i in 1 2 3; do
    mysql -h0 -P9306 -e "INSERT INTO test_concurrent (id, title) VALUES ($((1000+i)), 'Concurrent insert $i')" 2>&1 &
done
wait
mysql -h0 -P9306 -e "SELECT COUNT(*) FROM test_concurrent WHERE id >= 1001 AND id <= 1003"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_auto_dims (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -E -e "SHOW CREATE TABLE test_auto_dims"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_rapid (
    content_text TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content_text'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
for i in $(seq 1 20); do
    mysql -h0 -P9306 -e "REPLACE INTO test_rapid (id, content_text) VALUES (1, 'Iteration $i')" 2>/dev/null
done
mysql -h0 -P9306 -e "SELECT content_text FROM test_rapid WHERE id=1"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_row (
    title TEXT,
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, content'
)"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_empty_row (id, content) VALUES (1, 'Content without title')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_empty_row WHERE KNN(vec, 1, 'test query')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_row (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_compare_col (
    title TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_row (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "INSERT INTO test_compare_col (id, title) VALUES (1, 'Machine learning algorithms')"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_row WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id FROM test_compare_col WHERE KNN(vec, 1, 'machine learning')"
––– output –––
OK
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_columnar (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='columnar'" --load="INSERT INTO test_scale_columnar (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_columnar"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_columnar;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_columnar ORDER BY id ASC LIMIT 100" > /tmp/test_scale_columnar.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_columnar WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
manticore-load --quiet --json --init="CREATE TABLE test_scale_rowwise (
    id BIGINT,
    title TEXT,
    description TEXT,
    category STRING ATTRIBUTE,
    age INT,
    salary FLOAT,
    discount FLOAT,
    is_active BOOL,
    last_login TIMESTAMP,
    product_codes MULTI,
    large_values MULTI64,
    additional_info JSON,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='title, description'
) engine='rowwise'" --load="INSERT INTO test_scale_rowwise (id, title, description, category, age, salary, discount, is_active, last_login, product_codes, large_values, additional_info) VALUES (<increment>, 'Document title <increment>', 'Document description <increment>', 'cat_<increment>', <int/18/65>, <float/1000.0/100000.0>, <float/0.0/50.0>, <int/0/1>, <int/1609459200/1704067200>, (<array/1/5/100/999>), (<array/1/3/1000000/9999999>), '{\"rating\":<int/1/5>,\"verified\":<int/0/1>}')" --batch-size=500 --threads=2 --total=5000
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT COUNT(*) as total FROM test_scale_rowwise"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "DESCRIBE test_scale_rowwise;"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT * FROM test_scale_rowwise ORDER BY id ASC LIMIT 100" > /tmp/test_scale_rowwise.txt
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "SELECT id, title, category, knn_dist() FROM test_scale_rowwise WHERE KNN(vec, 5, 'Document title 100') LIMIT 5\G"
––– output –––
*************************** 1. row ***************************
        id: 10012
     title: Document title 10011
  category: cat_10009
knn_dist(): 0.28152#!/[0-9]{3}/!#
*************************** 2. row ***************************
        id: 10020
     title: Document title 10019
  category: cat_10017
knn_dist(): 0.28869#!/[0-9]{3}/!#
*************************** 3. row ***************************
        id: 10052
     title: Document title 10051
  category: cat_10049
knn_dist(): 0.31627#!/[0-9]{3}/!#
*************************** 4. row ***************************
        id: 10072
     title: Document title 10071
  category: cat_10069
- knn_dist(): 0.33028#!/[0-9]{3}/!#
+ knn_dist(): 0.330283
*************************** 5. row ***************************
        id: 10044
     title: Document title 10043
  category: cat_10041
knn_dist(): 0.33115#!/[0-9]{3}/!#
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_batch_perf (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s)
mysql -h0 -P9306 -e "INSERT INTO test_batch_perf (id, content) VALUES
(1, 'First text'), (2, 'Second text'), (3, 'Third text'),
(4, 'Fourth text'), (5, 'Fifth text')"
END_TIME=$(date +%s)
echo "Batch insert time: $((END_TIME - START_TIME)) seconds"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_row (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='rowwise'"; echo $?
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE perf_test_col (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'"; echo $?
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_row (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Row-wise 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
START_TIME=$(date +%s%N)
for i in {1..10}; do
    mysql -h0 -P9306 -e "INSERT INTO perf_test_col (id, content) VALUES ($i, 'Test content $i')" 2>/dev/null
done
END_TIME=$(date +%s%N)
echo "Columnar 10 inserts: $(( (END_TIME - START_TIME) / 1000000 )) ms"
––– output –––
OK
––– input –––
mysql -h0 -P9306 -e "CREATE TABLE test_empty_search (
    content TEXT,
    vec FLOAT_VECTOR KNN_TYPE='hnsw' HNSW_SIMILARITY='l2'
    MODEL_NAME='sentence-transformers/all-MiniLM-L6-v2'
    FROM='content'
) engine='columnar'";
mysql -h0 -P9306 -e "SELECT id FROM test_empty_search WHERE KNN(vec, 5, 'search query')"; echo $?
––– output –––
OK

sanikolaev and others added 17 commits March 21, 2026 11:50

ci: change DOCS_PER_CHUNK to constexpr, otherwise it won't compile in…

cc00e97

… some cases

chore: misc/build-libs-for-test-kit.sh

9e8b71c

post-review fixes

a42941c

more post-review fixes

978b374

more post-review fixes

5e1684a

feat(embeddings): support unlimited timeout for API models

3879f7b

- api_timeout=0 now means unlimited instead of 10s default - Jina, OpenAI, Voyage models accept None for no timeout - Update FFI test with api_url parameter

style(embeddings): format struct field declarations

2eff35c

- Apply consistent multi-line formatting to error enum variants - Simplify timeout duration mapping using Option::map - Remove redundant match expressions in model constructors

chore(deps): bump manticoresearch to 0ed9a0d

302509f

feat(embeddings): add ONNX model support with candle-onnx integration

a8e2443

- Integrate candle-onnx for ONNX model inference - Enable local ONNX model loading and execution - Add support for local embeddings using ONNX models

Merge remote-tracking branch 'origin' into feature/onnx

f3dfcf5

chore(deps): update candle to manticoresoftware fork

df47004

- Update candle to manticoresoftware fork - Switch candle dependencies to git revision

ci(workflows): add protoc installation to embedding builds

5ea3de7

- Add protoc tool installation to embedding CI workflow - Install before main build in Linux job - Install conditionally for non-Linux distribution builds

ci(embedding): add platform-specific BLAS acceleration features

cb9949c

perf(embeddings): optimize release profile for speed over size

19bc71d

perf(embeddings): batch tokenize texts with encode_batch

7ba8b47

donhardman force-pushed the fix/performance branch from d247bfa to 7ba8b47 Compare March 30, 2026 07:08

donhardman changed the base branch from master to feature/onnx March 30, 2026 07:09

donhardman changed the base branch from feature/onnx to master March 30, 2026 09:19

donhardman added 2 commits March 30, 2026 13:31

Merge branch 'master' into fix/performance

596c6de

perf(embeddings): use repeat_n for tensor padding

19e1144

- Replaced std::iter::repeat().take() with std::iter::repeat_n() in BertEmbeddingModel and OnnxEmbeddingModel - repeat_n is more efficient as it avoids intermediate iterator overhead - Removed unused candle_onnx import

donhardman requested a review from sanikolaev March 30, 2026 11:51

donhardman added 5 commits March 30, 2026 16:09

ci: fix zstd removal timing for cache compatibility

612ebe5

ci(workflows): switch to curl for downloads

26e63b5

ci(win_test): extract diskc/winbundle from sysroot tarball

c6cef3b

donhardman added 5 commits March 30, 2026 22:57

chore: increment LIB_VERSION to 13

c267c1d

chore(knn): decrement LIB_VERSION from 13 to 12

4d6ac58

ci(test): add debug commands to check library dependencies

4ec3d3e

ci(embedding): install oneMKL for hgemm support

d260b9b

ci(embedding): force static OpenMP for oneMKL

545a3f3

donhardman added 3 commits April 2, 2026 16:26

ci(embedding): bump rust and remove mkl setup

4d46546

- Bump rust-min-libc images to version 1.94.1 - Remove manual oneMKL installation for amd64 builds

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Boost the performance of the inference#146

Boost the performance of the inference#146
donhardman wants to merge 33 commits intomasterfrom
fix/performance

donhardman commented Mar 26, 2026

Uh oh!

github-actions bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 30, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

donhardman commented Mar 26, 2026

Uh oh!

github-actions bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linux debug test results

Uh oh!

github-actions bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Linux release test results

Uh oh!

github-actions bot commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Windows test results

Uh oh!

github-actions bot commented Apr 2, 2026

clt

Failed tests:

Uh oh!

github-actions bot commented Apr 2, 2026

clt

Failed tests:

Uh oh!

github-actions bot commented Apr 2, 2026

clt

Failed tests:

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Mar 30, 2026 •

edited

Loading

github-actions bot commented Mar 30, 2026 •

edited

Loading

github-actions bot commented Mar 30, 2026 •

edited

Loading