7.4.2
- fix(core): remove table 'lock' file on drop by @ideoma in https://github.com/questdb/questdb/pull/4341
- fix(sql): fix incorrect results returned when executing lower/upper/substring functions in parallel by @puzpuzpuz in https://github.com/questdb/questdb/pull/4337
- fix(sql): SAMPLE BY not to mix multiplication asterisk with wildcard column by @glasstiger in https://github.com/questdb/questdb/pull/4351
- perf(core): improve hash table lookup performance for small string keys such as column names and symbols by @puzpuzpuz in https://github.com/questdb/questdb/pull/4348
- fix(core): rare segfault during o3 commit of a non-wal table by @jerrinot in https://github.com/questdb/questdb/pull/4354
- fix(sql): fix error in some CASE statements when ELSE branch is missing by @bluestreak01 in https://github.com/questdb/questdb/pull/4367
- feat(core): optimise memory usage to reduce OS kills by @ideoma in https://github.com/questdb/questdb/pull/4368
- perf(ilp): optimize ingestion for tables with large number of columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/4371
- fix(ilp): fix 'maximum buffer size exceeded' in ILP client by @jerrinot in https://github.com/questdb/questdb/pull/4372
- fix(sql): bug for negative upper bound (second argument) in BETWEEN operator by @sivukhin in https://github.com/questdb/questdb/pull/4375
- feat(ilp): sender config string can disable individual flush types separately by @jerrinot in https://github.com/questdb/questdb/pull/4385
- feat(ilp): client configuration string supports auto_flush_bytes by @jerrinot in https://github.com/questdb/questdb/pull/4391
- fix(sql): constant in explicit group by should not remove parallel optimisation by @nwoolmer in https://github.com/questdb/questdb/pull/4384
- fix(core): fix potential error rebuilding nascent data index by @bluestreak01 in https://github.com/questdb/questdb/pull/4383
- perf(sql): speed up SQL COPY and REST API CSV import by @puzpuzpuz in https://github.com/questdb/questdb/pull/4381
- feat(sql): add bind variable support for long256 and IPv4 operators by @puzpuzpuz in https://github.com/questdb/questdb/pull/4282
- feat(sql): add milliseconds and microseconds support to datediff by @javier in https://github.com/questdb/questdb/pull/4398
- feat(sql): add milliseconds and microseconds support to dateadd by @javier in https://github.com/questdb/questdb/pull/4396
- fix(http): client to process data received before TLS close_notification by @jerrinot in https://github.com/questdb/questdb/pull/4397
- fix(sql): fix incorrect results returned by GROUP BY queries with int or symbol key by @puzpuzpuz in https://github.com/questdb/questdb/pull/4353
- perf(chore): optimize binary search corner cases by @sivukhin in https://github.com/questdb/questdb/pull/4402
- fix(sql): fix incorrect results for parallel group by with symbol like filter by @puzpuzpuz in https://github.com/questdb/questdb/pull/4411
- fix(sql): internal error when inserting null string as a designated timestamp by @jerrinot in https://github.com/questdb/questdb/pull/4408
- fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in https://github.com/questdb/questdb/pull/4420
- fix(core): fix possible issue with incorrect partition size after update by @bluestreak01 in https://github.com/questdb/questdb/pull/4412
- fix(sql): fix SAMPLE BY queries with functions in SELECT clause by @ideoma in https://github.com/questdb/questdb/pull/4404
- fix(sql): FILL(LINEAR) now supports ALIGN TO CALENDAR by @nwoolmer in https://github.com/questdb/questdb/pull/4392
- fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in https://github.com/questdb/questdb/pull/4420
- fix(sql): fix inaccurate double comparison by @bluestreak01 and @eugenels in https://github.com/questdb/questdb/pull/4417
- @hoangsvit made their first contribution in https://github.com/questdb/questdb/pull/4358
- @sivukhin made their first contribution in https://github.com/questdb/questdb/pull/4375
Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.2
7.4.1
- fix(core): remove table 'lock' file on drop by @ideoma in https://github.com/questdb/questdb/pull/4341
- fix(sql): fix incorrect results returned when executing lower/upper/substring functions in parallel by @puzpuzpuz in https://github.com/questdb/questdb/pull/4337
- fix(sql): SAMPLE BY not to mix multiplication asterisk with wildcard column by @glasstiger in https://github.com/questdb/questdb/pull/4351
- perf(core): improve hash table lookup performance for small string keys such as column names and symbols by @puzpuzpuz in https://github.com/questdb/questdb/pull/4348
- fix(core): rare segfault during o3 commit of a non-wal table by @jerrinot in https://github.com/questdb/questdb/pull/4354
- fix(sql): fix error in some CASE statements when ELSE branch is missing by @bluestreak01 in https://github.com/questdb/questdb/pull/4367
- feat(core): optimise memory usage to reduce OS kills by @ideoma in https://github.com/questdb/questdb/pull/4368
- perf(ilp): optimize ingestion for tables with large number of columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/4371
- fix(ilp): fix 'maximum buffer size exceeded' in ILP client by @jerrinot in https://github.com/questdb/questdb/pull/4372
- fix(sql): bug for negative upper bound (second argument) in BETWEEN operator by @sivukhin in https://github.com/questdb/questdb/pull/4375
- feat(ilp): sender config string can disable individual flush types separately by @jerrinot in https://github.com/questdb/questdb/pull/4385
- feat(ilp): client configuration string supports auto_flush_bytes by @jerrinot in https://github.com/questdb/questdb/pull/4391
- fix(sql): constant in explicit group by should not remove parallel optimisation by @nwoolmer in https://github.com/questdb/questdb/pull/4384
- fix(core): fix potential error rebuilding nascent data index by @bluestreak01 in https://github.com/questdb/questdb/pull/4383
- perf(sql): speed up SQL COPY and REST API CSV import by @puzpuzpuz in https://github.com/questdb/questdb/pull/4381
- feat(sql): add bind variable support for long256 and IPv4 operators by @puzpuzpuz in https://github.com/questdb/questdb/pull/4282
- feat(sql): add milliseconds and microseconds support to datediff by @javier in https://github.com/questdb/questdb/pull/4398
- feat(sql): add milliseconds and microseconds support to dateadd by @javier in https://github.com/questdb/questdb/pull/4396
- fix(http): client to process data received before TLS close_notification by @jerrinot in https://github.com/questdb/questdb/pull/4397
- fix(sql): fix incorrect results returned by GROUP BY queries with int or symbol key by @puzpuzpuz in https://github.com/questdb/questdb/pull/4353
- perf(chore): optimize binary search corner cases by @sivukhin in https://github.com/questdb/questdb/pull/4402
- fix(sql): fix incorrect results for parallel group by with symbol like filter by @puzpuzpuz in https://github.com/questdb/questdb/pull/4411
- fix(sql): internal error when inserting null string as a designated timestamp by @jerrinot in https://github.com/questdb/questdb/pull/4408
- fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in https://github.com/questdb/questdb/pull/4420
- fix(core): fix possible issue with incorrect partition size after update by @bluestreak01 in https://github.com/questdb/questdb/pull/4412
- fix(sql): fix SAMPLE BY queries with functions in SELECT clause by @ideoma in https://github.com/questdb/questdb/pull/4404
- fix(sql): FILL(LINEAR) now supports ALIGN TO CALENDAR by @nwoolmer in https://github.com/questdb/questdb/pull/4392
- fix(sql): fix stuck parallel group by query due to incorrect query timeout handling by @puzpuzpuz in https://github.com/questdb/questdb/pull/4420
- fix(sql): fix inaccurate double comparison by @bluestreak01 and @eugenels in https://github.com/questdb/questdb/pull/4417
- @hoangsvit made their first contribution in https://github.com/questdb/questdb/pull/4358
- @sivukhin made their first contribution in https://github.com/questdb/questdb/pull/4375
Full Changelog: https://github.com/questdb/questdb/compare/7.4.0...7.4.1
7.4.0
Ahh, Spring! Sunshine, flowers, all that nice stuff.
As the wheels of the seasons turn, so to the wheels of the applications we're all busy building.
Hopefully good things bloom for us all.
QuestDB is pleased to release version 7.4!
As you've come to expect, this release further improves core database performance. Also as per usual, several bugs and rough edges have been addressed. In important addition, there are note-able breaking changes that will provide a smoother overall experience and set us up well for future changes. Please address them prior to upgrade, and reach out to us if you run into any issues.
-
SAMPLE BY
queries now useALIGN TO CALENDAR
by default. As such, if you're usingSAMPLE BY
and expectingALIGN TO FIRST OBSERVATION
, you'll need to update your queries to specify that syntax. -
Java client: The ILP Sender used in our first-party Java client now requires an explicit selection of Transport, (HTTP or TCP) for enhanced reliability and clarity in data ingestion workflows. You will need to pass this value forward. See the PR for more information. If you do not use the Java client, but use another language client, see your respective language client repository and update if needed.
-
CREATE TABLE AS
andINSERT INTO SELECT
statements now operate non-atomically and are batched by default. If you require an atomic table, you can now use the very powerful sounding:CREATE ATOMIC TABLE
.
This release improves parallel GROUP BY
query performance and also makes SAMPLE BY
queries more efficient. In addition, we've reduced thread-local allocator overhead in single-threaded GROUP BY
& SAMPLE BY
queries. The first/last aggregate functions for string columns have also been optimized.
- Resolved issues leading to segmentation faults and incorrect results in certain SQL queries involving GROUP BY and SAMPLE BY clauses.
- Addressed a bug that could cause a table to be omitted from snapshots after significant DDL changes.
- Fixed HTTP request handling to prevent rejections when content-length is set to 0.
- Solved potential TLS handshake stalls on MacOS.
- Corrected the behavior of altering tables to add indexes that could inadvertently remove deduplication keys.
- Eliminated a crash during ingestion and various other stability issues.
- fix(sql): potential incorrect results in non-keyed parallel GROUP BY query by @puzpuzpuz in https://github.com/questdb/questdb/pull/4234
- fix(sql): segfault when running SAMPLE BY query with index-based filter and last/first functions by @puzpuzpuz in https://github.com/questdb/questdb/pull/4233
- fix(core): fix an issue where a table was not being included in the snapshot due to significant DDL changes being applied. by @ideoma in https://github.com/questdb/questdb/pull/4245
- fix(http): fix rejects of HTTP GET requests when content-length set to 0 by @ideoma in https://github.com/questdb/questdb/pull/4242
- fix(core): TLS handshake might occasionally get stuck on MacOS by @jerrinot in https://github.com/questdb/questdb/pull/4269
- fix(sql): fix incorrect unordered map creation check by @puzpuzpuz in https://github.com/questdb/questdb/pull/4255
- perf(sql): parallel GROUP BY support for first/last functions by @puzpuzpuz in https://github.com/questdb/questdb/pull/4267
- fix(sql): fix altering table to add index can remove dedup key by @ideoma in https://github.com/questdb/questdb/pull/4270
- feat(sql): parallel
sample by
implementation by @bluestreak01 in https://github.com/questdb/questdb/pull/4259 - perf(sql): speed up first/last aggregate functions for string columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/4276
- fix(sql): fix a bug in indexed symbol comparison by @mtopolnik in https://github.com/questdb/questdb/pull/4273
- fix(core): fix wal table is suspended after invalid drop partition SQL by @ideoma in https://github.com/questdb/questdb/pull/4272
- fix(core): fix double-free in text loader by @bluestreak01 in https://github.com/questdb/questdb/pull/4278
- perf(sql): avoid redundant parsing of UUID bind variables by @puzpuzpuz in https://github.com/questdb/questdb/pull/4281
- feat(core): support WAL sequencer transaction file partitioned format by @ideoma in https://github.com/questdb/questdb/pull/4194
- feat(sql): breaking change💥 - change default behaviour of SAMPLE BY to ALIGN TO CALENDAR instead of ALIGN TO FIRST OBSERVATION by @nwoolmer in https://github.com/questdb/questdb/pull/4277
- fix(core): fix possible crash during ingestion (fuzz) by @mtopolnik in https://github.com/questdb/questdb/pull/4286
- fix(sql): qualified names in ORDER BY join queries no longer cause AssertionError by @nwoolmer in https://github.com/questdb/questdb/pull/4274
- perf(sql): remove thread-local allocator access overhead in single-threaded GROUP BY/SAMPLE BY queries by @puzpuzpuz in https://github.com/questdb/questdb/pull/4271
- feat(ilp): breaking change💥 - ILP Sender requires explicit Transport selection by @jerrinot in https://github.com/questdb/questdb/pull/4293
- fix(core): fix ignored JVM arguments in AMI by @petr-tichy in https://github.com/questdb/questdb/pull/4258
- fix(http): send http timeout response instead of disconnect on long csv export by @ideoma in https://github.com/questdb/questdb/pull/4300
- feat(sql): breaking change💥 - change default behaviour of CREATE TABLE AS/INSERT INTO SELECT statements to be non-atomic and batched by @nwoolmer in https://github.com/questdb/questdb/pull/4287
- fix(sql): fix potentially incorrect values produced by LPad/RPad functions by @eugenels in https://github.com/questdb/questdb/pull/4327
- fix(sql): fix ILIKE for upper case non-ASCII patterns by @puzpuzpuz in https://github.com/questdb/questdb/pull/4321
- fix(sql): fix occasional failures with first()/last()/first_not_null()/last_not_null() direct string function by @jerrinot in https://github.com/questdb/questdb/pull/4324
- @petr-tichy made their first contribution in https://github.com/questdb/questdb/pull/4258
Full Changelog: https://github.com/questdb/questdb/compare/7.3.10...7.3.11
7.3.10
Fresh from QuestDB 🐲
The 2024 Lunar New Year marks the Year of the Dragon. 🐉 When you think of the mighty dragon, what comes to mind? For us, well, we're a little different. Dragons evoke strength, wisdom, and - naturally - wild performance (we're performance obsessed).
Within that theme, the 7.3.10 release offers a 5x-10x speed-up in ASOF
and LT JOIN
queries, as well as a number of optimizations in parallel filters, GROUP BY
, and within the JIT compiler. Further, UUID
and LONG256
columns are now supported by parallel GROUP BY
for efficient use of all hardware resources. For workloads that feature small, but frequent transactions you'll also see improved performance.
Familiar with InfluxDB? Interested in using InfluxDB Line Protocol (ILP), but with something more robust under-the-hood? This release provides 5x faster ingestion performance when multiple connections are in use, further increasing overall throughput strength compared to InfluxDB, especially in data with high cardinality. Whether you're looking to leverage ILP for high performance streaming, or upgrading an existing InfluxDB workload over to QuestDB, this update makes it easier.
Read more about "drop-in" InfluxDB migration in our recent blog.
Alongside the usual battery of bug fixes, this release also includes a new HyperLogLog-based approx_count_distinct()
SQL function that is 3x-5x faster and more memory efficient than the existing count_distinct()
function for high cardinality data sets. And finally, Write-Ahead Log (WAL) enabled tables are now the default table type. This means that the benefits of WAL & WAL-required features like deduplication are available to each newly created table.
Whether you're new to QuestDB, working in prod, or migrating existing time series workloads, this release has something for you.
Performance improvements
- perf(sql): support uuid and long256 in parallel GROUP BY by @nwoolmer in https://github.com/questdb/questdb/pull/4140
- perf(sql): avoid redundant disk reads in ORDER BY + LIMIT queries with parallel filter by @puzpuzpuz in https://github.com/questdb/questdb/pull/4148
- perf(sql): use JIT compiled filter in parallel GROUP BY by @puzpuzpuz in https://github.com/questdb/questdb/pull/4138
- perf(sql): speed up GROUP BY and JOIN in case of very small fixed-size keys by @puzpuzpuz in https://github.com/questdb/questdb/pull/4134
- perf(sql): add JIT compilation support for string/binary null check by @mtopolnik in https://github.com/questdb/questdb/pull/4177
- perf(sql): improve parallel group by detection when aliasing columns by @nwoolmer in https://github.com/questdb/questdb/pull/4190
- perf(sql): avoid full table scan in ASOF and LT JOIN with no additional columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/4199
- perf(ilp): fix ingress perf issue for small, but frequent transactions by @bluestreak01 in https://github.com/questdb/questdb/pull/4221
New features
- feat(ilp): make ILP over HTTP transactional for single table by @ideoma in https://github.com/questdb/questdb/pull/4110
- feat(ilp): ILP client supports HTTP as a transport by @jerrinot in https://github.com/questdb/questdb/pull/4111
- feat(ilp): Influx Line Protocol client configuration via strings by @jerrinot in https://github.com/questdb/questdb/pull/4166
- feat(conf): add new configuration settings to server.conf (including Enterprise settings) by @amunra in https://github.com/questdb/questdb/pull/4175
- feat(sql): add approx_count_distinct() SQL function by @piotrrzysko in https://github.com/questdb/questdb/pull/4083
- feat(core): enable WAL tables by default by @bluestreak01 in https://github.com/questdb/questdb/pull/4223
Fixes and other improvements
- fix(http): fix CSV import for files larger than 2GB by @puzpuzpuz in https://github.com/questdb/questdb/pull/4132
- fix(http): fix HTTP protocol errors following non-successful requests by @ideoma in https://github.com/questdb/questdb/pull/4125
- fix(core): fix database stuck on startup after some table drops by @ideoma in https://github.com/questdb/questdb/pull/4152
- fix(core): fix wal resume exception after table suspended because of an error by @ideoma in https://github.com/questdb/questdb/pull/4162
- fix(core): communicate posix_fallocate's return value via errno by @eugenels in https://github.com/questdb/questdb/pull/4158
- fix(sql): fix exception when running insert as select by @ideoma in https://github.com/questdb/questdb/pull/4157
- fix(core): fix invalid table writer state after ALTER TABLE SQUASH PARTITIONS by @puzpuzpuz in https://github.com/questdb/questdb/pull/4153
- fix(core): disable query cancellation and timeout in WAL apply job by @bziobrowski in https://github.com/questdb/questdb/pull/4176
- fix(sql): fix NullPointerException in LATEST ON with nested query by @puzpuzpuz in https://github.com/questdb/questdb/pull/4192
- fix(core): fix errors / data inconsistencies after restoring from a snapshot by @ideoma in https://github.com/questdb/questdb/pull/4205
- fix(sql): enable parallel group-by on servers with fewer than 4 cores by @nwoolmer in https://github.com/questdb/questdb/pull/4208
- fix(core): fix snapshot restore to correctly restore tables when snapshot is taken under heavy DDL load by @ideoma in https://github.com/questdb/questdb/pull/4212
- fix(pgwire): fix a memory leak when using prepared statement and UUID columns by @jerrinot in https://github.com/questdb/questdb/pull/4214
New Contributors
- @mtopolnik made their first contribution in https://github.com/questdb/questdb/pull/4177
Full Changelog: https://github.com/questdb/questdb/compare/7.3.9...7.3.10
7.3.8b
Welcome back from the holidays! (If you had one...)
In the latest QuestDB release, we've focused on enhanced stability and introduced two innovative features, currently in beta. Notably, we've improved GROUP BY
performance tenfold through parallel SQL executions and introduced InfluxDB Line Protocol (ILP) support over HTTP, which is compatible with existing InfluxDB drivers. Those familiar with ILP will find a clean entryway into QuestDB.
Thanks to @ssharaev and @pompeiifreckles for their first contributions - we appreciate you.
Dive in to explore the release in detail. 🤿
What's changed
Please note that the following features are in
beta
- please let us know if you run into any issues.
- 10x
GROUP BY
performance improvement via parallel SQL executions. Should you encounter any problems with this implementation, you can safely disabled[cairo.sql.parallel.groupby.enabled=false]
it via configuration without downgrading the database instance. - InfluxDB Line Protocol (ILP) support over HTTP. This implementation supports existing InfluxDB drivers and eases the transition into QuestDB for existing ILP users. The new client has control over transaction boundaries and will offer helpful feedback on transaction activity. We're also pleased to offer no noticeable performance impact in ingestions performance compared to our TCP implementation.
New features
- feat(sql): add sign() functions by @bziobrowski in https://github.com/questdb/questdb/pull/4050
- feat(ilp): ILP over HTTP, InfluxDB native clients writing support, ILP feedback by @ideoma in https://github.com/questdb/questdb/pull/4034
- feat(sql): add
nullif(double, double)
SQL function by @ssharaev in https://github.com/questdb/questdb/pull/4063 - perf(sql): parallel group by with optional filtering by @puzpuzpuz in https://github.com/questdb/questdb/pull/4032
- feat(sql): add query registry and query cancellation by @bziobrowski in https://github.com/questdb/questdb/pull/4066
Fixes and improvements
- fix(sql): fix handling of keyed string aggregation of first(), first_not_null(), last(), last_not_null() functions by @bziobrowski in https://github.com/questdb/questdb/pull/4068
- fix(core): fix non-existing partition drop can suspend WAL table by @ideoma in https://github.com/questdb/questdb/pull/4064
- fix(ilp): ilp@http small fixes by @glasstiger in https://github.com/questdb/questdb/pull/4077
- fix(sql): implement 'case expression when value' for float and double data types by @bziobrowski in https://github.com/questdb/questdb/pull/4046
- fix(sql): fix NullPointerException in replace() function with symbol arguments by @bziobrowski in https://github.com/questdb/questdb/pull/4073
- fix(core): fix tables() function permanently returning error after some DDLs by @ideoma in https://github.com/questdb/questdb/pull/4081
- fix(core): fix potential and unlikely numeric overlflow that could lead to incorrect query results by @pompeiifreckles in https://github.com/questdb/questdb/pull/4093
- fix(sql): fix potential races when regexp_replace function is used in where or group by clause by @puzpuzpuz in https://github.com/questdb/questdb/pull/4102
- fix(core): fix non-partitioned table corruption caused by
truncate
& table corruption after restore from snapshot by @bluestreak01 in https://github.com/questdb/questdb/pull/4095 - fix(core): fix edge case error when importing a large CSV file @jerrinot in https://github.com/questdb/questdb/pull/4103
New Contributors
- @ssharaev made their first contribution in https://github.com/questdb/questdb/pull/4063
- @pompeiifreckles made their first contribution in https://github.com/questdb/questdb/pull/4093
Full Changelog: https://github.com/questdb/questdb/compare/7.3.7...7.3.8b
7.3.7
What's Changed
The hits keep comin'. Newly released QuestDB 7.3.7 adds supporting functions to pair with our recently released window functions. It also offers brings further improvements to SQL and ingestion performance - this has been a pattern of late!
Additional key features include:
- A 5x speed up of WAL Apply for small transactions
- New built-in functions for covariance, correlation, cumulative sum, and first value window
- Enhanced compatibility with Grafana
Thanks to @AjCH1 and @ksankeerth for their first contributions!
New features
- feat(core): 5x speed up of WAL Apply for small transaction and add SQL function to read WAL transactions metadata by @ideoma in https://github.com/questdb/questdb/pull/3992
- perf(sql): micro optimizations SqlKeywords implementation (short-circuit evaluation, inline variable) by @nitram509 in https://github.com/questdb/questdb/pull/3978
- feat(sql):
information_schema.columns()
function to provide list of columns across all tables in the database by @marregui in https://github.com/questdb/questdb/pull/3975 - feat(sql): add covariance and corr built-in functions by @charlespnh in https://github.com/questdb/questdb/pull/4012
- feat(sql): add cumulative sum and first value window funtions by @bziobrowski in https://github.com/questdb/questdb/pull/4001
- fix(core): fix failed SQL update not suspending WAL table potentially leading to a data loss by @ideoma in https://github.com/questdb/questdb/pull/4015
- perf(sql): execute complex sum aggregates on short column in parallel by @puzpuzpuz in https://github.com/questdb/questdb/pull/4024
- perf(sql): optimize order by designated timestamp and other columns with limit by @bziobrowski in https://github.com/questdb/questdb/pull/4025
- perf(sql): speed up select distinct symbol queries by @puzpuzpuz in https://github.com/questdb/questdb/pull/4030
Fixes and improvements
- fix(sql): fix symbol != value filter not applied by @bziobrowski in https://github.com/questdb/questdb/pull/4035
- fix(sql): fix edge case internal error in case-switch expression by @jerrinot in https://github.com/questdb/questdb/pull/4033
- fix(sql): fix column propagation for queries with set operations and distinct by @bziobrowski in https://github.com/questdb/questdb/pull/4028
- fix(core): include system tables into database snapshots by @puzpuzpuz in https://github.com/questdb/questdb/pull/4026
- fix(sql): improve compatibility with grafana by @bziobrowski in https://github.com/questdb/questdb/pull/4039
- fix(sql): occasionally failing queries when under rapid structural changes by @jerrinot in https://github.com/questdb/questdb/pull/4000
- fix(sql): fix parsing of single character window frame bounds by @bziobrowski in https://github.com/questdb/questdb/pull/4045
New Contributors
- @AjCH1 made their first contribution in https://github.com/questdb/questdb/pull/3384
- @ksankeerth made their first contribution in https://github.com/questdb/questdb/pull/4008
Full Changelog: https://github.com/questdb/questdb/compare/7.3.5...7.3.7
7.3.6
What's Changed
Release highlights are new window functions and SQL and ingestion performance improvements.
New features
- feat(core): 5x speed up of WAL Apply for small transaction and add SQL function to read WAL transactions metadata by @ideoma in https://github.com/questdb/questdb/pull/3992
- perf(sql): micro optimizations SqlKeywords implementation (short-circuit evaluation, inline variable) by @nitram509 in https://github.com/questdb/questdb/pull/3978
- feat(sql):
information_schema.columns()
function to provide list of columns across all tables in the database by @marregui in https://github.com/questdb/questdb/pull/3975 - feat(sql): add covariance and corr built-in functions by @charlespnh in https://github.com/questdb/questdb/pull/4012
- feat(sql): add cumulative sum and first value window funtions by @bziobrowski in https://github.com/questdb/questdb/pull/4001
- fix(core): fix failed SQL update not suspending WAL table potentially leading to a data loss by @ideoma in https://github.com/questdb/questdb/pull/4015
- perf(sql): execute complex sum aggregates on short column in parallel by @puzpuzpuz in https://github.com/questdb/questdb/pull/4024
- perf(sql): optimize order by designated timestamp and other columns with limit by @bziobrowski in https://github.com/questdb/questdb/pull/4025
- perf(sql): speed up select distinct symbol queries by @puzpuzpuz in https://github.com/questdb/questdb/pull/4030
Fixes and improvements
- fix(sql): fix symbol != value filter not applied by @bziobrowski in https://github.com/questdb/questdb/pull/4035
- fix(sql): fix edge case internal error in case-switch expression by @jerrinot in https://github.com/questdb/questdb/pull/4033
- fix(sql): fix column propagation for queries with set operations and distinct by @bziobrowski in https://github.com/questdb/questdb/pull/4028
- fix(core): include system tables into database snapshots by @puzpuzpuz in https://github.com/questdb/questdb/pull/4026
- fix(sql): improve compatibility with grafana by @bziobrowski in https://github.com/questdb/questdb/pull/4039
- fix(sql): occasionally failing queries when under rapid structural changes by @jerrinot in https://github.com/questdb/questdb/pull/4000
- fix(sql): fix parsing of single character window frame bounds by @bziobrowski in https://github.com/questdb/questdb/pull/4045
New Contributors
- @AjCH1 made their first contribution in https://github.com/questdb/questdb/pull/3384
- @ksankeerth made their first contribution in https://github.com/questdb/questdb/pull/4008
Full Changelog: https://github.com/questdb/questdb/compare/7.3.5...7.3.6
7.3.5
What's Changed
Following up our recent 7.3.4
release which provided Window Functions, Web Console improvements, and more, we're pleased to announce another significant release. Headlining 7.3.5
are the following key additions:
- Percentile function: Find an approximate percentile for a given series with
approx_percentile(DD)
. - Optimized queries: QuestDB has long stood out for its high performance ingestion. Not be left behind, queries also keep getting faster. In benchmarks, recent performance optimizations have shown 50% improvement to several query types.
New features
- feat(core): config to limit RSS memory used for writing by @ideoma in https://github.com/questdb/questdb/pull/3937
- feat(core): print insufficient Linux kernel limits on start-up by @puzpuzpuz in https://github.com/questdb/questdb/pull/3980
- feat(sql): implement approx_percentile function by @kevinmingtarja in https://github.com/questdb/questdb/pull/3900
- feat(sql): string_agg function support in keyed group by queries by @puzpuzpuz in https://github.com/questdb/questdb/pull/3991
- feat(sql): add first_not_null and last_not_null functions for GeoHash and IPv4 data types by @bziobrowski in https://github.com/questdb/questdb/pull/3929
- feat(sql) add
show server_version
by @nitram509 in https://github.com/questdb/questdb/pull/3901 - feat(sql): add
show parameters
SQL by @bluestreak01 in https://github.com/questdb/questdb/pull/3972 - perf(sql): optimize count_distinct symbol function by @puzpuzpuz in https://github.com/questdb/questdb/pull/3974
Fixes and improvements
- fix(sql): fix segfault in aggregate vectorized functions on zero input length by @puzpuzpuz in https://github.com/questdb/questdb/pull/3944
- fix(sql): maintain timestamp metadata in window factories by @bziobrowski in https://github.com/questdb/questdb/pull/3940
- fix(core): introduce separate settings for WAL writer pool by @puzpuzpuz in https://github.com/questdb/questdb/pull/3948
- fix(sql): improve count(*) and limit x,y performance by @bziobrowski in https://github.com/questdb/questdb/pull/3951
- fix(sql): fix create table like with dedup by @brunocalza in https://github.com/questdb/questdb/pull/3964
- fix(sql): fix several small sql issues by @bziobrowski in https://github.com/questdb/questdb/pull/3970
- fix(core): fix UPDATE timeout can leave non-WAL table in half updated state by @ideoma in https://github.com/questdb/questdb/pull/3946
- fix(core): reduce disk and memory footprint of system tables by @puzpuzpuz in https://github.com/questdb/questdb/pull/3945
- fix(core): recognize XFS file system as supported by @puzpuzpuz in https://github.com/questdb/questdb/pull/3968
- fix(core): handling of unexpected errors in SQL execution by @glasstiger in https://github.com/questdb/questdb/pull/3963
7.3.4
What's Changed
Better. Faster. Stronger. And even more good looking. 😎
Highlighting our latest release:
-
Window functions. These flexible functions provide moving averages, running totals, or Top-N results within a group, and more via a "window", defined by an
OFFSET
against aRANGE
orROWS
. Unlike aggregate functions, window functions do not cause rows to become grouped into a single output row — the rows retain their separate identities. Additional perks? Performance and an easing of query complexity. -
More functions. We've added:
first(boolean)
,last(boolean)
,first(string)
,last(string)
, mathematical exponentexp(D)
andfirst_not_null()
,last_not_null()
, which operate on most nullable data types, except IPv4 and GeoHash. -
Performance. Our goal is top performance.
LIKE
/ILIKE
queries now perform over 80% faster. -
Web Console Refresh. The best tools provide the help that you need without getting in your way. We've tuned the Web Console UI to put what you need closer to where you need it. Less used UI elements have had their significance reduced and overall navigation has been streamlined. On top of that, a discrete news feed will send you all the latest, so that you don't miss a beat.
Before:
After:
New features
- feat(sql): add first and last boolean aggregation functions by @rexf in https://github.com/questdb/questdb/pull/3873
- feat(sql): add math exp function by @rexf in https://github.com/questdb/questdb/pull/3903
- feat(sql): add first and last string aggregation functions by @rexf in https://github.com/questdb/questdb/pull/3879
- perf(sql): speed up simple LIKE/ILIKE filters by @puzpuzpuz in https://github.com/questdb/questdb/pull/3897
- feat(core): add metrics around worker by @marregui in https://github.com/questdb/questdb/pull/3872
- feat(web-console): general UI refresh, new News UI, new CSV Import UI #199
- feat(sql): sql functions to support traffic light widget in the Web Console by @bluestreak01 (#3918)
- feat(sql): add first_not_null and last_not_null aggregation functions for major data types by @rexf in https://github.com/questdb/questdb/pull/3883
- feat(sql): add vwap aggregation function by @rexf in https://github.com/questdb/questdb/pull/3885
- feat(sql): moving avg window function by @bluestreak01 @bziobrowski in https://github.com/questdb/questdb/pull/3763
- feat(sql): vectorized aggregation of
short
columns by @bluestreak01 (#3924)
UI
- feat(web-console): general UI refresh, new News UI, new CSV Import UI https://github.com/questdb/ui/pull/199
Improvements and bug fixes
- fix(http): REST interface sends HTTP error replies on runtime errors by @jerrinot (#3908)
- fix(sql): fix duplicate column error for implicit timestamp by @bziobrowski #3921
- fix(sql): prevent predicate pushdown into subquery with window clause by @bziobrowski (#3923)
- fix(sql): fix segmentation fault in non-keyed max(int) vectorised implementation by @bziobrowski (#3914)
- fix(sql): fix server hang on evaluating large integer addition expression by @bziobrowski (#3922)
- fix(sql): fix 'Invalid column' error for columns used in non-equality join conditions by @bziobrowski (#3912)
- fix(core): fix random file descriptor leak on WAL table drop by @ideoma (#3919)
- fix(core): fix connection error to HTTP TCP port on MacOS by @jerrinot in https://github.com/questdb/questdb/pull/3830
- fix(http): prometheus exporter occasionally fails to send all metrics by @jerrinot in https://github.com/questdb/questdb/pull/3835
- fix(sql): ensure EXPLAIN (FORMAT JSON) returns valid JSON by @biosfood in https://github.com/questdb/questdb/pull/3756
- fix(sql): fix ArrayIndexOutOfBoundsException in select distinct with duplicate columns by @puzpuzpuz in https://github.com/questdb/questdb/pull/3834
- fix(core): fix intermittent writing failures by @jerrinot in https://github.com/questdb/questdb/pull/3844
- fix(core): fix write error appearing after symbol column drop by @ideoma in https://github.com/questdb/questdb/pull/3876
- fix(sql): rename column from
name
totable_name
in cursor function tables by @marregui in https://github.com/questdb/questdb/pull/3857 - fix(sql): fix sql error [col in (1,2)] when col is type short by @ideoma in https://github.com/questdb/questdb/pull/3887
- fix(http): large prometheus metrics responses split across multiple chunks by @amunra in https://github.com/questdb/questdb/pull/3843
- fix(sql): fix error on UPDATE statement with WAL tables after ALTER by @ideoma in https://github.com/questdb/questdb/pull/3894
- fix(sql): fix comparing Date columns with date literals by @jerrinot in https://github.com/questdb/questdb/pull/3862
- fix(core): add FreeBSD UFS (0x35) to the list of supported file systems by @nitram509 in https://github.com/questdb/questdb/pull/3899
- fix(sql): fix symbol value comparison with <, <=, >, >= operators by @bziobrowski in https://github.com/questdb/questdb/pull/3909
- fix(sql): fix modulo int operator result with nulls by @bziobrowski in https://github.com/questdb/questdb/pull/3910
Documentation
- docs(core): add section about the cloud to Spanish README by @javier in https://github.com/questdb/questdb/pull/3831
New Contributors
- @abhiram6121 made their first contribution in https://github.com/questdb/questdb/pull/3789
- @biosfood made their first contribution in https://github.com/questdb/questdb/pull/3756
- @rexf made their first contribution in https://github.com/questdb/questdb/pull/3873
- @nitram509 made their first contribution in https://github.com/questdb/questdb/pull/3899
Full Changelog: https://github.com/questdb/questdb/compare/7.3.3...7.3.4
7.3.3
Overview
Thanks for all who contributed to our most recent release.
This release provides a set of stability fixes and usability improvements.
We are excited to highlight the following areas:
-
Improved operations: A trio of features including improved snapshots to replace existing backup functionality, connection count gauge metrics and a new
mkdir.mode
for detached partitions make QuestDB easier to operate. -
InfluxDB Line Protocol improvements: The InfluxDB Line Protocol is commonly used for rapid ingest. We’ve made it better, by introducing more flexible and less error prone timestamps. In addition, it also authenticates faster and disconnects on error when using WAL tables.
-
New aggregate functions: Thanks to community member @charlespnh QuestDB now offers
variance()
,var_samp()
andvar_pop()
,stddev()
,stddev_pop()
functions.
What is new
CORE
- allow using SNAPSHOT statements as an alternative to BACKUP by @puzpuzpuz in https://github.com/questdb/questdb/pull/3761
- add connection count gauge metrics for all network interfaces by @puzpuzpuz in https://github.com/questdb/questdb/pull/3815
- add
mkdir.mode
permission config for detached partitions by @debanjanc01 in https://github.com/questdb/questdb/pull/3818
SQL
- allow string args for ipv4 operators by @allegraharris in https://github.com/questdb/questdb/pull/3665
- add alias and population standard deviation by @charlespnh in https://github.com/questdb/questdb/pull/3731
- add variance built-in functions by @charlespnh in https://github.com/questdb/questdb/pull/3770
ILP
- introduce less error-prone timestamp methods in Java ILP client by @puzpuzpuz in https://github.com/questdb/questdb/pull/3755
What is fixed
CORE
- prevent connection reset during inactivity by @jerrinot in https://github.com/questdb/questdb/pull/3734
- fix incorrect metadata pool sizing, which could cause ingestion performance to degrade by @bluestreak01 in https://github.com/questdb/questdb/pull/3762
- fix race leading to occasionally frozen WAL transaction processing on some tables by @puzpuzpuz in https://github.com/questdb/questdb/pull/3785
- workaround for broken file locking on Docker Desktop on MacOS by @jerrinot in https://github.com/questdb/questdb/pull/3797
- fix wal apply suspending table on edge condition by @ideoma in https://github.com/questdb/questdb/pull/3808
- fix transient select error after symbol column rename sequence by @ideoma in https://github.com/questdb/questdb/pull/3787
- use the same GC algorithm in Docker image as with other QuestDB distros by @debanjanc01 in https://github.com/questdb/questdb/pull/3819
SQL
- sql keywords uses as table metadata are required to be enclosed in double quotes by @bluestreak01 in https://github.com/questdb/questdb/pull/3737
- fix empty result on second run of query with group by and limit by @bziobrowski in https://github.com/questdb/questdb/pull/3754
- fix error on trailing semicolon in ALTER TABLE RESUME WAL by @puzpuzpuz in https://github.com/questdb/questdb/pull/3774
- fix wrong results returned from union queries with similar joins by @puzpuzpuz in https://github.com/questdb/questdb/pull/3768
- fix NOT IN expression parsing with function arguments by @bziobrowski in https://github.com/questdb/questdb/pull/3813
- fix json plan redundant double quote by @RustyLunaRover in https://github.com/questdb/questdb/pull/3804
- fix table name case sensitivity in SQL UPDATE by @hui-jie-lim in https://github.com/questdb/questdb/pull/3820
- fix parsing of indexed symbol predicates in where clause by @bziobrowski in https://github.com/questdb/questdb/pull/3810
- improve order by clause propagation by @bziobrowski in https://github.com/questdb/questdb/pull/3822
ILP
- ILP disconnects on error with WAL tables by @jerrinot in https://github.com/questdb/questdb/pull/3758
- faster ILP authentication by @jerrinot in https://github.com/questdb/questdb/pull/3817
HTTP
- http min works even when the main http server is disabled by @jerrinot in https://github.com/questdb/questdb/pull/3732
PGWire
- fix pg-wire protocol errors caused by network IO fragmentation by @ideoma in https://github.com/questdb/questdb/pull/3751
- fix pgwire transaction flag not cleaned on context reuse by @ideoma in https://github.com/questdb/questdb/pull/3777
New Contributors
- @heysagnik made their first contribution in https://github.com/questdb/questdb/pull/3698
- @charlespnh made their first contribution in https://github.com/questdb/questdb/pull/3731
- @debanjanc01 made their first contribution in https://github.com/questdb/questdb/pull/3818
- @RustyLunaRover made their first contribution in https://github.com/questdb/questdb/pull/3804
- @hui-jie-lim made their first contribution in https://github.com/questdb/questdb/pull/3820
Full Changelog: https://github.com/questdb/questdb/compare/7.3.2...7.3.3