questdb/questdb
 Watch   
 Star   
 Fork   
简介统计版本
26 days ago
questdb
questdb

8.3.3

What's Changed

Full Changelog: https://github.com/questdb/questdb/compare/8.3.2...8.3.3

2025-05-24 05:06:52
questdb
questdb

8.3.2

8.3.2 is a stability release, bringing a number bugfixes and improvements. There have been improvements to materialized views, checkpoints, zfs support, and instance metrics, and of course, our built-in web console!

Materialized Views can now be configured to refresh on a timed schedule, instead of immediately whenever new data is written. You can also modify the TTL and refresh limit post-creation, giving you more flexibility to optimise current and new views.

Taking checkpoints will now consume much less disk space, and we've also added some new metrics to help you to monitor your overall database's health, and be alerted to writing delays or suspended tables.

This release will shortly be followed by the first beta release of the new ARRAY type. This is a multi-dimensional array (think NumPy), which will initially support DOUBLE values. This will be accompanied by the first upgardes for the ILP clients, to move from the text protocol to a bespoke binary protocol. This means arrays can be sent to the database efficiently, without expansion into lots of text, and is the first step towards supporting all of QuestDB's rich features through the database clients.

For any questions or feedback, please join us on Slack or on Discourse.

See also our prettier release notes page.

Breaking changes 💥

  • The greatest and least functions will now only return a null if all of the inputs are null, instead of any. This aligns the behaviour with MSSQL.

  • The WITHIN function now defaults to a different implementation, which runs WITHIN before LATEST BY.

    • If you were using the old implementation, which ran LATEST BY first, you can re-enable it using query.within.latest.by.optimisation.enabled=true. This variant was only used for WHERE + LATEST BY + indexed symbols, and no other time.
    • Or alternatively, use the new implementation and a subquery to force the LATEST BY to run first.

Highlights

UI

  • The web console now allows you to set an instance's name and colour! This is handy when switching between different deployments and instances, to make sure you run your queries on the correct console.
  • The query log UI element has been upgraded, fixing a few snags, and now supports horizontal scrolling for long rows.
  • (Enterprise) SSO users will not have to log back in every time they log out; instead, an existing SSO session can be resumed, or you can switch to a different account.

Prometheus Metrics

New metrics have been added to help monitor the performance of tier-1 (WAL) and tier-2 (Table) storage:

  • suspended_tables; this is a global counter for how many tables are currently suspended.
  • wal_apply_seq_txn; this is a global counter for sequencerTxn, the txn number corresponding to the latest transaction written to WAL, for a particular table.
  • wal_apply_writer_txn; this is a global counter for writerTxn, the txn number corresponding to the latest transaction visible for read, for a particular table.
  • You can subtract the above two numbers and track this over time, to estimate if the database is backing up and falling behind on data visibility.
  • You can also track replication lag by comparing metrics between the primary and the replica. More granular information can be obtained by polling wal_tables() and sys.telemetry_wal;

Materialized Views

  • Materialized Views can now be refreshed on a timer schedule.
    • CREATE MATERIALIZED VIEW price_1h REFRESH START '2025-09-12T00:00:00.000000Z' EVERY 1h AS ...
    • You can also alter existing views to swap to a new scheduled refresh:
    • ALTER MATERIALIZED VIEW price_1h SET REFRESH START '2035-09-12T00:00:00.000000Z' EVERY 1d;
  • You can now add a TTL post-creation:
    • ALTER MATERIALIZED VIEW view_name SET TTL 42 DAYS;
  • You can now add a refresh limit post-creation, which bounds how far back an incremental refresh will go
    • ALTER MATERIALIZED VIEW view_name SET REFRESH LIMIT 12 HOURS;
  • Base table names are no longer case-sensitive.

Checkpointing

  • The storage cost to have an active checkpoint has been dramatically reduced. This means you are less likely to run out of storage if you have out-of-order (O3) writes whilst CHECKPOINT CREATE is active.
    • If you encounter any issues, you can roll back to the old version using cairo.txn.scoreboard.format=1.
    • This change also fixes max txn in-flight errors, which could be caused by a leaked query.

Geospatial

  • We have upgraded the WITHIN operator to work for generic WHERE clauses, instead of just LATEST BY queries. This should make it much easier to filter for data by geohash location.

Changelist

2025-04-28 19:48:26
questdb
questdb

8.3.1

8.3.1 is a stability release, and moves Materialized Views into GA. Thanks to all the early-adopters who have helped us with crucial feedback, to make sure they are efficient and robust. Give them a try!

8.3.1 also brings the first SQL Hints, which will give you more control over your query execution plans. For now, only one hint is supported - USE_ASOF_BINARY_SEARCH - which allows you to swap between two algorithms for handling filtered ASOF JOIN queries. Next on the list - controlling when the database chooses to use an INDEX scan!

For any questions or feedback, please join us on Slack or on Discourse!

See also our prettier release notes page!

Changelist

Full Changelog: https://github.com/questdb/questdb/compare/8.3.0...8.3.1

2025-04-17 02:53:00
questdb
questdb

8.3.0

8.3.0 moves Materialized Views into an advanced beta. Thanks to all of you have tried the beta and helped us iron out the remaining creases!

This release also brings significant ingestion performance improvements, particularly for small transactions and for batched PG Wire inserts. Not to mention a variety of bug fixes and safety enhancements to improve overall user experience.

But let's not forget about querying data! Check out how your ASOF JOIN queries are doing! We have released dramatic performance improvements for ASOF JOINs that include interval scans and/or post-join filters. Benching against larger datasets has shown 100-1000x speedups for some queries.

For any questions or feedback, please join us on Slack or on Discourse!

See also our prettier release notes page!

Breaking changes 💥

  • Materialized Views will now be enabled by default. If you do not wish to use them, you can disable them by setting cairo.mat.view.enabled=false.

  • Designated timestamp columns can no longer hold values beyond the Year 9999. We found that these values were generally produced by mistaken arithmetic. This prevents undroppable partition issues, and also reduces the chances of a configured TTL dropping your data.

  • Some IPv4 built-in functions were removed. You may need to introduce explicit casts in SQL queries that relied on implicit conversions between ipv4 and other database types. This removes some ambiguity that was causing issues with queries unrelated to ipv4.

  • Converting char to numeric values now aligns with PostgreSQL behaviour, whether implicitly or explicitly casted.

  • SAMPLE BY will now correctly handle a variety of DST-related timestamp grouping issues. SAMPLE BY queries that use DST shifts are more likely to be executeed in parallel, too. These changes mean your result set might look different to before - but should be a consistent, rational output.

  • FLOAT and DOUBLE types, when output to JSON or web console, will now have different precision. Previously, this was limited artificially. Now, it will print as many digits as it can. If you rely on the old output for any integration testing, you may need to upgrade your tests!

Highlights

  • Materialized Views are now in an advanced BETA.
    • Views will be correctly invalidated when the base table is dropped.
    • Syntax errors in materialized view definitions should report the correct error position.
    • PARTITION BY and parentheses () are now optional.
    • Full mat view refreshes should no longer trigger occassional OOM errors.
    • Still to come: a mat view write-performance enhancement, which is also a pre-step to supporting time-range DELETEs!
  • ASOF JOIN queries that include filters over large datasets will now run 100-1000x faster than before!
  • SYMBOL columns can now have their capacity changed in-place, using ALTER TABLE table ALTER COLUMN column SYMBOL CAPACITY 4096!
  • Ingestion of small batches has been sped by up to 100x!
  • Ingestion via PG Wire batched inserts is faster and more memory efficient.

Changelist

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/8.2.3...8.3.0

2025-04-17 00:58:18
questdb
questdb

8.3.0

8.3.0 moves Materialized Views into an advanced beta. Thanks to all of you have tried the beta and helped us iron out the remaining creases!

This release also brings significant ingestion performance improvements, particularly for small transactions and for batched PG Wire inserts. Not to mention a variety of bug fixes and safety enhancements to improve overall user experience.

But let's not forget about querying data! Check out how your ASOF JOIN queries are doing! We have released dramatic performance improvements for ASOF JOINs that include interval scans and/or post-join filters. Benching against larger datasets has shown 100-1000x speedups for some queries.

For any questions or feedback, please join us on Slack or on Discourse!

See also our prettier release notes page!

Breaking changes 💥

  • Materialized Views will now be enabled by default. If you do not wish to use them, you can disable them by setting cairo.mat.view.enabled=false.

  • Designated timestamp columns can no longer hold values beyond the Year 9999. We found that these values were generally produced by mistaken arithmetic. This prevents undroppable partition issues, and also reduces the chances of a configured TTL dropping your data.

  • Some IPv4 built-in functions were removed. You may need to introduce explicit casts in SQL queries that relied on implicit conversions between ipv4 and other database types. This removes some ambiguity that was causing issues with queries unrelated to ipv4.

  • Converting char to numeric values now aligns with PostgreSQL behaviour, whether implicitly or explicitly casted.

  • SAMPLE BY will now correctly handle a variety of DST-related timestamp grouping issues. SAMPLE BY queries that use DST shifts are more likely to be executeed in parallel, too. These changes mean your result set might look different to before - but should be a consistent, rational output.

  • FLOAT and DOUBLE types, when output to JSON or web console, will now have different precision. Previously, this was limited artificially. Now, it will print as many digits as it can. If you rely on the old output for any integration testing, you may need to upgrade your tests!

Highlights

  • Materialized Views are now in an advanced BETA.
    • Views will be correctly invalidated when the base table is dropped.
    • Syntax errors in materialized view definitions should report the correct error position.
    • PARTITION BY and parentheses () are now optional.
    • Full mat view refreshes should no longer trigger occassional OOM errors.
    • Still to come: a mat view write-performance enhancement, which is also a pre-step to supporting time-range DELETEs!
  • ASOF JOIN queries that include filters over large datasets will now run 100-1000x faster than before!
  • SYMBOL columns can now have their capacity changed in-place, using ALTER TABLE table ALTER COLUMN column SYMBOL CAPACITY 4096!
  • Ingestion of small batches has been sped by up to 100x!
  • Ingestion via PG Wire batched inserts is faster and more memory efficient.

Changelist

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/8.2.3...8.3.0

2025-03-21 00:33:37
questdb
questdb

8.2.3

Highlights

  • Materialized views in BETA
  • Relaxed GLIBC requirement (2.28 aarch64 and 2.17 amd64) to allow QuestDB run on older Linux distros
  • new dense_rank() window function
  • new approx_median() function
  • Performance optimisation 10-30x for value in (select column... and select distinct SQLs

What's Changed

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/8.2.2...8.2.3

2025-01-29 22:41:41
questdb
questdb

8.2.2

Our first release of 2025! 🎉

What's Changed

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/8.2.1...8.2.2

2024-12-02 19:57:43
questdb
questdb

8.2.1

Winter is coming... And so are more features and improvements within 8.2.1.

This release follows a restive holiday weekend and the successful deployment of 8.2.0. Our prior release shipped our re-constructed PostgreSQL server implementation. After seeing the initial usage patterns, we've rolled out a series of stability fixes to smooth the overall journey. Those consuming PGWire, within tools like R Studio and PowerBI, will find a more seamless experience.

And, as usual, a series of fixes and performance improvements and general optimizations.

Breaking change 💥

  • The dateadd(AIN) function will now error when provided with an invalid period or n (stride) value, if they are constants.
    • Use the errors to reconstruct the function syntax as appropriate.
  • PowerBI integration now uses PostgreSQL type names in information_schema.columns().
    • Please update to these type names if you've applied an alternative.

Web Console 📈

Additions

  • Support for ID token in Auth ui/#355
  • News Image zoom, to better see all the cool new stuff :) ui/#350

Changes

  • Highlight integer numbers that include _ (underscore) separator ui/#353
  • Handle and display error received while scrolling the grid ui/#348
  • Use new information_schema.questdb_columns() instead of information_schema.columns() ui/#359

New features 🐣

  • New min(D) and max(D) window function.

Performance improvements 🚀

  • Reduced CPU idle load for improved overall efficiency.
  • Improved length(varchar) function performance.

Bug fixes 🐛

  • Fixed an issue when purging dropped tables that previously led to critical failures.
  • Fixed issue where WAL tables would be suspended when a table update was followed by a rename.
  • Fixed error reporting in the scenario where a 'drop table if exists' SQL command does not find the table.
  • Fixed incorrect error reporting to PostgreSQL drivers:
    • Prepared statements that might fail at runtime will no longer contain duplicate error messages
    • SQL compilation errors will not be reported as NPE
  • Modified INSERT AS SELECT will correctly return the number of rows inserted.

Pull requests

Full Changelog: https://github.com/questdb/questdb/compare/8.2.0...8.2.1

2024-11-22 21:55:14
questdb
questdb

8.2.0

This release brings the PostgresSQL server rewrite to comply with the wire protocol rigorously. As a result, we now support PowerBI and R to name a couple of tools. There will be more to come!

  • speed up small transactions via PGwire by ~10x, added latency control to the background job, which improves data visibility
  • New SQL functions: count() window function, md5, sha1, sha256, and regr_intercept
  • Query performance boost for SQL Window Functions.
  • The healthcheck endpoint is now more responsive, which improves server stability
  • More error messages returned to HTTP client rather than being just logged

QuestDB Enterprise new features:

  • Microsoft EntraID compatibility
  • Improved support for Pingfederate

What's Changed

What's Changed

Ent:

feat(core): support for decoding groups from ID token feat(core): replace admin-only checks with permissions fix(core): fix for not being able to login with other users if built-in admin is disabled feat(core): config switch to disable ILP over TCP authentication

New Contributors

Full Changelog: https://github.com/questdb/questdb/compare/8.1.4...8.2.0

2024-10-23 21:36:08
questdb
questdb

8.1.4

It's spooky season. 🎃

What scares us the most? Poor performance. 😱

And just behind that, bugs and unintuitive UIs. Aaaahhhh!!

This release brings light to all of the aforementioned fears, as we continue with our seasonal theme of balance.

New features 🐣

Slick Web Console polish

Last patch, we introduced tabs to the Web Console experience. With tabs, navigating and work flows are easier and cleaner.

The polish continues in 8.1.4 and includes many small, tidy incremental improvements such as:

  • More prominent error highlighting
  • Prettier formatting when errors are generated by larger queries
  • Revamped the look and feel of the table list panel — gone is the pop-up menu
  • Once executed via shortcut, such as F9 or CTL + Enter, hitting the same shortcut will no longer cancel running SQL
  • Indicates Running... when a query is running (... so you'd better go catch it!)
  • While Running... the "play" button is replaced with "stop" button
  • Removed = as word delimiter, so that expressions like == can be highlighted as errors

And more! Download, upgrade or checkout our live demo to see them.

Regression function regr_slope()

We've introduced the regr_slope(y, x) SQL function, which calculates the slope of the linear regression line determined by the dependent variable y and independent variable x. One might use such a function in statistical analysis, trend identification, and predictive modelling:

SELECT regr_slope(sales, time) FROM revenue_data;

This query computes the slope of sales over time. Great for trend spotting.

Support for COUNT(DISTINCT col) and STRING_AGG(DISTINCT col)

The DISTINCT keyword is now available within aggregate functions to operate on unique values.

Counting unique values:

SELECT count(DISTINCT user_id) FROM events;

Aggregating unique strings:

SELECT string_agg(DISTINCT city, ', ') FROM customers;

This simplifies queries where you need to work with distinct entries. We all appreciate more concise and expressive SQL.

Performance improvements 🚀

  • Suped up many statistical functions with parallelized execution. Functions like var_samp(), var_pop(), stddev_samp(), stddev_pop(), covar_samp(), covar_pop(), and corr() are now up to 10× faster.

  • Reduced memory usage during paralleled aggregate queries

General improvements and fixes 🔧

  • Addressed a disk space leak issue when upserting with deduplication

  • Resolved phantom errors in SELECT queries caused by concurrent partition squashing

  • Fixed inconsistencies in the interaction between the Web Console and the static metadata cache

  • Fixed issues causing unexpected timeouts and error messages in parallel query execution

  • Resolved an infinite loop issue that could occur on errors in parallel filter and GROUP BY queries - as fun as it was...

  • Addressed a potential transaction leak in table reader/writer initialization on ZFS.

  • Fixed a rare failure to restore from a checkpoint, triggered by an indexed column being created while a checkpoint is in progress.

  • Unified macOS runtime requirements to macOS 13 for build consistency.

  • Fixed checkpoint backward compatibility with snapshot semantics.

  • Fixed a memory leak with pipelined UPDATEs in the PostgreSQL Wire Protocol.

Pull requests

Full Changelog: https://github.com/questdb/questdb/compare/8.1.2...8.1.4