gildas-lormeau/zip.js
 Watch   
 Star   
 Fork   
23 hours ago
zip.js

v2.8.60

What's Changed in v2.8.60

New features

  • New VERSION constant exposing the version of the library at runtime (e.g. "2.8.60"). It matches the version declared in package.json; the continuous integration verifies the agreement
  • New getRegisteredCodecs() function. It returns the definitions of the codecs registered with registerCodec(), as snapshots that cannot alter the registry. The CompressionStream and DecompressionStream classes of a codec registered with codecURI appear in the result once its module has been imported
  • New getSupportedCompressionMethods() function. It returns the compression methods supported in the current environment and configuration: the built-in methods resolved against the compression streams available at the time of the call, followed by the registered codecs. Each entry reports the compression and decompression support separately, e.g. Deflate64 is read-only. The support of a codec registered with codecURI only is reported as undefined until its module is imported
  • Registered compression codecs now receive the size of the source data as CompressionStreamOptions#uncompressedSize when the reader has a known size. Codecs such as Zstandard can use it as the pledged source size and include the content size in the compressed frame (#675)

Bug fixes

  • The zip-fs-core build now exports the full core API. It previously exported only the filesystem classes, so configure(), registerCodec(), the reader and writer classes, and the constants were unreachable from this build

Documentation

  • The Reader class documents how to implement random access to files opened with the runtime APIs, with a Deno example
  • The offset and usdz options are documented as read when the ZipWriter is created and ignored when passed to ZipWriter#add, and the default value of offset read from Writer#size is documented

Tests and continuous integration

  • New tests cover the registered codec snapshots, the codec constructor options transmitted to web workers, the VERSION constant, and the supported compression methods including the deferred resolution of codecURI codecs
  • The version bump now rebuilds the bundles so the published files embed the version, and the continuous integration verifies that the version constant agrees with the declared versions

Credits

  • Thanks to @xqdoo00o for implementing the uncompressedSize option of the compression codecs (#675)
  • Claude (Fable 5) contributed to every other change listed above
2 days ago
zip.js

v2.8.59

What's Changed in v2.8.59

New features

  • New ZipReader#warnings property and warnings property on entries. They report non-fatal anomalies noticed while reading, as an array of { reason, filename? } objects deduplicated by reason. ZipReader#warnings is replaced on each getEntries() call and collects the archive-level observations: an unsorted central directory, an unknown "version needed to extract", the compressed patched data bit, a malformed extra field, unknown zip64 extensible data, and a wrapped 16-bit entry count. The entry-level warnings property is set by getData() and collects the local file header observations. The checks controlled by the strictness option deposit a warning with the same reason when a lower strictness tolerates what "strict" rejects: appended or prepended data, trailing central directory data, duplicate filenames, a mismatched zip64 end of central directory record, and local file header mismatches. The warnings only report bytes the parse already read, so enabling nothing costs no additional I/O. The reasons are exported as 14 WARNING_* constants
  • New isZipFile() function. It returns true if the data looks like a zip file, i.e. if ZipReader#getEntries called on the same data would locate the archive structure. It runs the same end-anchored search as ZipReader and verifies that a central directory record is stored where the end of central directory record points, without parsing the entries. The strictness and maxAppendedDataSize options control the tolerated appended data with the same semantics and defaults as ZipReader
  • New centralExtraField option of ZipWriter#add. It sets an extra field written only in the central directory record, complementing the localExtraField option which targets the local file header and the extraField option which targets both

Behavior changes

  • Leading and trailing whitespace in entry names is now preserved by ZipWriter#add instead of being silently trimmed. The zip specification does not restrict whitespace in filenames; note that Windows filesystems cannot represent a trailing space or dot in a name
  • Unclaimed bytes lying between the last central directory record and the end of central directory record are now detected, even when the declared central directory size matches the records. The "strict" strictness rejects such archives with the ERR_AMBIGUOUS_ARCHIVE error and the lower strictness levels deposit the "trailing central directory data" warning. These bytes were previously accepted silently at every strictness level, although the gap can hide records that other readers interpret, e.g. an unadvertised zip64 end of central directory record, and Info-ZIP and 7-Zip both flag such files. The check is skipped when the central directory is encrypted, because the plaintext is legitimately shorter than the stored data

Tests and continuous integration

  • A new test suite covers the warnings: each reason is triggered by byte surgery on a well-formed zip file and asserted both as a warning at the tolerant levels and as a rejection at the levels that make the corresponding check throw
  • New regression tests lock the preserved whitespace in entry names, the isZipFile() probe, the centralExtraField option, and the detection of unclaimed bytes before the end of central directory record
2 days ago
zip.js

v2.8.58

What's Changed in v2.8.58

New features

  • New ZipWriter#appendZip method. It copies the entries of an existing zip file into the current zip. Unlike prependZip, it can be called at any position: after entries have been added, between add() calls, and repeatedly to merge several zip files. The central directory of the copied file is rebuilt and its entries are relocated to the positions they get in the output. prependZip is kept as a deprecated alias
  • New rawLastModDate option of ZipWriter#add. It sets the raw MS-DOS date and time of the entry directly, which passThrough copies of ZipCrypto entries need (see below)
  • New localDirectory.dataOffset property. It is the byte offset of the entry data, i.e. the entry offset plus the size of the local file header, of the filename and of the extra field. It can be used with Reader#createReadable to serve ranged requests into an entry stored without compression
  • New ERR_ZIP_CRYPTO_LAST_MOD_DATE error constant

Behavior changes

  • Errors of add() and appendZip() calls left un-awaited are not lost anymore. close() waits for the pending calls and throws the first unreported error, with all of them available in its entryErrors property. Throwing counts as reporting: catching the error and calling close() again finalizes the zip file without the failed entries. ZipWriterStream now aborts its writable when an entry fails, so the readable errors instead of hanging
  • An interrupted appendZip() copy now sets hasCorruptedEntries on the writer and keeps the offsets of the entries written after it consistent
  • The "version needed to extract" field is now 10 for entries stored without compression or encryption, instead of 20
  • Last modification dates before 1980 are now clamped to the MS-DOS epoch instead of underflowing the date field
  • The directory property of read entries is now derived from the trailing slash of the filename alone. A name ending with "/" is a folder even when the entry declares an uncompressed size
  • Unicode Path and Unicode Comment extra fields are now applied only when their version is 1, as required by section 4.6.8 of the zip specification
  • Reading an archive with a multiple of 65,536 entries and no zip64 record now returns all the entries. The 16-bit count of the end of central directory record wraps around; the reader detects the wrap by walking the central directory records past the declared count. The recovery is skipped when the strictness checks reject ambiguous archives
  • The unsafe* optimizations of the minifier were removed from the builds. Two of them shipped real miscompilations in the past, one of which stayed undetected for four years, and the size they saved was about 50 bytes per compressed bundle

Bug fixes

  • ZipCrypto entries copied with passThrough can now be read back with their password. The password verification byte of ZipCrypto depends on the raw date of the entry when a data descriptor is used, so a copy that regenerated the date or forced the descriptor failed with ERR_INVALID_PASSWORD. The dataDescriptor option is not forced anymore for pass-through ZipCrypto data, the new rawLastModDate option preserves the raw date, and the filesystem API forwards both when exporting, throwing the new ERR_ZIP_CRYPTO_LAST_MOD_DATE error if the date is overridden
  • The end of central directory records of split zip files now declare the number of central directory entries stored on the last disk, as required by section 4.4.21 of the zip specification, instead of the total. The count is 0 when the record starts on a fresh disk, which is how Info-ZIP fills the field. The check deciding whether the disk number of the record requires zip64 also accounts for the actual record and comment length when predicting a disk rollover near 65,535 disks
  • The spanning signature of split zip files is now written while holding the writer lock. A first entry written with bufferedWrite, or interleaved un-awaited add() calls, could fail on the locked stream or misplace the signature
  • Duplicate filenames are now detected before the entry waits for a worker slot. Two add() calls with the same filename made while the worker pool was saturated could both be accepted

Documentation

  • The usdz option states that its constraints apply to the entries written with add() only. The entries copied with appendZip keep the layout of the source zip file and are not checked
  • The ZipReader constructor states that a stream input is buffered entirely in memory, because reading a zip file requires random access, and points at custom Reader implementations for large seekable resources
  • WritableWriter#size states that a value set before the first write is used as the starting offset
  • useUnicodeFileNames states that disabling it only clears the language encoding flag and does not re-encode the filenames
  • passThrough documents the coupling between ZipCrypto and the last modification date
  • msDosCompatible documents how PKUNZIP handles folder entries

Tests and continuous integration

  • New PKZIP 1.10 and PKZIP 2.04g fixtures (attributes, comments, spanned archives) with tests reading the archives produced by the original tools
  • New regression tests lock the fixes above: the spanning signature position with concurrent and buffered writes, the per-disk entry counts of spanning central directories, including in the golden output, the ZipCrypto pass-through copies, the queued duplicate names, and the retryable close()
  • The Safari suite retries up to four times, the ZipCrypto tests are immune to date and wrong-password flakes, and the test fixtures are resolved independently of the working directory
  • The dead extra field handling of the appendZip entry rebuild was removed, making it explicit that copied entries carry their extra fields verbatim
4 days ago
zip.js

v2.8.57

What's Changed in v2.8.57

New features

  • New ERR_UNSUPPORTED_UINT64 error constant

Behavior changes

  • Folders and empty stored entries are now written without a data descriptor. Their checksum and sizes are zero and known before the data is written, so bit 3 of the general purpose bit flag and the descriptor declared nothing, and other writers, e.g. Info-ZIP, leave them out. The local file header now carries the zeroed values directly, which shrinks every folder by the length of the descriptor. Those entries are also written directly instead of going through the buffered write path, since there is nothing to buffer. The descriptor is still written when the dataDescriptor option is set explicitly, and for encrypted entries
  • Reading an archive that declares a 64-bit value above Number.MAX_SAFE_INTEGER now throws the new ERR_UNSUPPORTED_UINT64 error. JavaScript numbers lose integer precision above 2^53 - 1, so a size or an offset that large was silently rounded to a nearby value and every computation derived from it was wrong. No valid archive is affected, such values describe contents beyond 8 PB

Bug fixes

  • The central directory records rebuilt by ZipWriter#prependZip now keep the zip64 layout of the source entries. The zip64 fields were selected again from the sizes of each entry, so a record whose source stored, e.g., only one of its sizes in the zip64 extra field was rebuilt with a different layout, and the zip64 field left in the raw extra field of the entry could be written twice. Prepending an archive and adding entries now produces the same bytes as writing all the entries directly
  • The InfoZip Unix extra field now stores both ids when only one of the uid and gid options is set, the missing one defaults to 0. The field used to declare the missing id with a length of 0, a layout Info-ZIP never writes and readers are not required to accept

Documentation

  • unixExtraFieldType now states that the filesystem API re-emits the uid and gid of imported entries as "infozip" whatever the field type found in the imported zip file, unless the option is set explicitly

Tests and continuous integration

  • The browser testing workflow was improved by @danny0838: the browser driver is determined automatically, the download URL of old Chromium versions is computed from the version, and the runner works on Windows (#674)
  • The web test runner now waits for the cleanup of the test frames before removing them and reports the in-flight tests when a run fails, and the old Chromium versions of the matrix are pinned to their snapshot position
  • The worker backpressure test now registers a codec that never signals backpressure and measures a deterministic bound on every runtime, instead of relying on the margin left by the native codec of each engine
  • New regression tests lock the fixes and changes above: the data descriptor rules for folders and empty stored entries, including the golden output, the zip64 layout of prepended entries, and the layout of the InfoZip Unix extra field when a single id is set
  • A new test locks that ZipWriter#remove returns false for an entry whose add() is still in flight and that the entry is written normally
4 days ago
zip.js

v2.8.56

What's Changed in v2.8.56

Bug fixes

  • The compression method stored in the WinZip AES extra field is no longer truncated to its low byte. The field declares the actual compression method of an encrypted entry in a 16-bit slot, and it was written with a single byte store. An encrypted entry using a method above 255, which a codec registered with registerCodec can use, announced a different method, so readers selected the wrong codec after decrypting. Registered codecs exist since v2.8.37
  • Writing an entry with the usdz option now throws ERR_INVALID_EXTRAFIELD_DATA when the extra field could exceed 64KB once the alignment padding is added. The padding is computed after the length check and adds up to 67 bytes, so an extra field close to the limit wrapped the 16-bit length field and produced a corrupt entry
  • The central directory written by ZipWriter#prependZip now points at the prepended entries when the writer has an initial offset. The offsets were computed from the source archive alone, so an archive written with the offset option, or into a writer already holding data, declared offsets short by that initial offset and the prepended entries could not be read back. The two features could be combined since v2.7.71
  • The digital signature record is now written within a single segment of a split archive. It could start at the end of one segment and continue in the next, unlike every other record, because only the end of central directory record checked the remaining space before being written. The writer now closes the disk first for the signature record as well. Signing is available since v2.8.47
  • ZipReader#digitalSignature is now defined on a split archive whose central directory starts on an earlier disk than the end of central directory record. The reader looked for the record in the bytes read for the central directory, which stop at the declared directory length in that case, so the signature was never found. The record is now read from the file when it does not follow the directory in those bytes
  • ZipDirectoryEntry#getExportedSize now throws ERR_UNDETERMINED_SIZE when the predicted size depends on the order the entries are written. The zip64 fields of an entry depend on its offset, so when the entries total more than 4GB and the write order is not guaranteed, e.g. keepOrder set to false, two orders can produce two sizes. The detection walked the entries in enumeration order, so it missed layouts where only another order crosses the threshold, and the exported archive could differ from the prediction. The prediction is available since v2.8.52
  • ZipDirectoryEntry#getExportedSize now honors the offset option. The option shifts the offsets stored in the central directory, which select the zip64 fields, and the prediction computed them from zero. Predicting an export with offset at 4GB or above returned a size short by the zip64 records the export actually writes

Documentation

  • CodecDefinition#codecURI now states that bundlers and single-file builds, e.g. deno compile, cannot follow the dynamic import of the codec module, so the module must be included explicitly, with deno compile --include or the equivalent option of the bundler

Tests and continuous integration

  • Every fix above is locked by a regression test: an encrypted round trip with a registered method above 255, a usdz entry with an extra field near the 64KB limit, prependZip into a writer with an initial offset, a sweep of segment sizes checking the placement and the read back of the digital signature record on split archives, and size predictions with order-dependent zip64 layouts and with the offset option
5 days ago
zip.js

v2.8.55

What's Changed in v2.8.55

New features

  • New ERR_UNDEFINED_COMPRESSION_METHOD error constant

Behavior changes

The two passThrough rules below change what the option accepts. Code that copies entries between archives by passing the compression method of the source entry, which is what the filesystem API does, is unaffected.

  • Writing an entry with the passThrough option now requires the compressionMethod option, and throws the new ERR_UNDEFINED_COMPRESSION_METHOD when it is missing. The data is copied as-is, so that option selects no codec, it declares how the data is already compressed and is written into the headers of the entry verbatim. It used to fall back to Deflate whatever the data was, so copying a stored entry without setting it produced an archive announcing Deflate over stored bytes, which no reader can decompress. Copying an entry read with ZipReader is a matter of passing its compressionMethod along. The entries with no content, e.g. the directories, ignore the option, as they ignore passThrough itself
  • The level option is now ignored for the entries written with passThrough. The data is never compressed, so the option describes nothing, and yet level set to 0 used to select the compression method written in the headers, stored instead of Deflate, and any level used to set the level bits of the general purpose bit flag. Set compressionMethod to declare how the data is compressed. level keeps applying to the other entries of the same archive, so exporting a filesystem with level set and passThrough set in the reader options still compresses the entries that were added to it and copies the entries that came from a zip file
  • The decryptCentralDirectory callback now receives the encrypted central directory alone. It used to be given the whole declared range of the directory, which also holds the digital signature record when the archive is signed, so the callback was handed bytes it cannot decrypt. The length is taken from the encryption header of the zip64 end of central directory record when it declares one, and falls back to the declared length of the directory. The callback has been given the whole range since it was introduced in v2.8.47

Bug fixes

  • Reading an archive whose central directory is encrypted no longer moves the directory somewhere else. The reader checks that the offset declared in the end of central directory record points at a central file header, and an encrypted directory carries no such signature, so the check failed and the offset was reconciled to another position. The offset is now trusted as well when the encryption header declares the size of the encrypted directory, and when the bytes at the offset look like an encrypted directory
  • The extensible data sector of the zip64 end of central directory record is now counted in the length of that record. The sector holds the encryption header of an encrypted central directory, and the reader subtracted the fixed length of the record only, so the computed end of the central directory sat past its real end by the length of the sector, and every offset derived from it was wrong
  • ZipReader#digitalSignature is now defined on an archive whose central directory is encrypted. The digital signature record follows the encrypted directory, so it is not part of what the decryption returns, and the reader looked for it in the decrypted bytes only
  • The local file headers masked by PKWARE strong encryption no longer make FileEntry#getData() throw ERR_AMBIGUOUS_ARCHIVE. When the central directory is encrypted, the local file header of an entry carries a placeholder filename and a zeroed checksum and sizes, and says so with bit 13 of its general purpose bit flag. The comparison against the central directory record that became the default in v2.8.53 rejected those archives. The filename, the checksum and the sizes are now left out of the comparison for those entries, the general purpose bit flag and the compression method are still compared
  • The published bundles no longer set the level bits of the general purpose bit flag on the entries written with the default compression level. index.min.js and the files of dist/ announced "super fast" on every Deflate entry whose level option was left unset, where the sources announce nothing. The minifier was configured with unsafe_comps, enabled in February 2022, which rewrites a comparison into its negation: !(0 > level) is true for an undefined level where level >= 0 is false. Those bits are advisory and no reader decompresses differently because of them, but an archive written by a bundle differed from the same archive written from the sources. The option is dropped from both minifier configurations, which costs 40 bytes on index.min.js

Documentation

  • ZipReader#digitalSignature now describes what the signature covers: the records of the central directory, read at ZipReader#directoryOffset, never including the digital signature record itself. zip.js does not verify signatures
  • ZipReader#directoryLength now warns that some writers, e.g. SecureZIP, count the digital signature record in the length they declare, so verifying the whole declared range can never succeed. Subtract 6 + digitalSignature.length from it when the record is stored inside the declared range
  • The passThrough option now describes how level and compressionMethod are treated, and states that the compression method is written into the headers as-is instead of selecting a codec
  • ZipReaderStream now states that it reads its input entirely into a Blob before it emits the first entry, since a zip file stores its central directory at the end. It is a convenience wrapper around ZipReader for stream sources, it does not extract the entries while the data is still arriving

Tests and continuous integration

  • New fixtures written by SecureZIP cover the strong encryption formats: AES-128, AES-192 and AES-256, stored and Deflate64 entries, an encrypted central directory, a certificate-based archive, a signed archive and an archive mixing encrypted and clear entries. They are read by the tests of decryptCentralDirectory and by the SecureZIP archive tests
  • A new fixture written by a third-party tool covers a zip64 entry whose sizes are stored in a data descriptor
  • The zip64 fixture of the HTTP test is rebuilt with real zip64 records, the previous one carried none although the test was named after them
  • The writer backpressure test now registers a codec that never signals backpressure on its writable side, instead of forcing the native one. It measured the invariant on Bun only, where the native CompressionStream behaved that way, and stopped measuring anything once Bun 1.4.0 fixed it. The registered codec keeps the test meaningful on every runtime
  • A new test locks the two passThrough rules above: the compression method is required, the level is ignored, and the level keeps applying to the other entries of the archive
6 days ago
zip.js

v2.8.54

What's Changed in v2.8.54

Breaking changes

  • ZipWriter#prependZip() now reads an array of readers as the disks of a split zip file, which is what an array denotes everywhere else in the API, and accepts a SplitDataReader instance the same way. The disks are read in order and the entries are relocated to the positions they get in the output. An array used to be concatenated and read as a single archive, which produced wrong offsets for a real split zip file. If you were passing an array of byte ranges of one zip file, concatenate them yourself and pass a single reader
  • ZipEntry#moveTo() is removed. It was deprecated and undeclared in the TypeScript definitions, and was a one-line alias of ZipFS#move(), which is the method to use
  • The undeclared ZipFS#addData() and ZipDirectoryEntry#addData() methods are removed. They were internal, never documented and never declared. The typed addText(), addBlob(), addUint8Array(), addData64URI(), addHttpContent(), addReadable(), addFile(), addFileSystemEntry() and addFileSystemHandle() methods cover what they did

Security

Both fixes below are reachable from an untrusted zip file read with ZipReader. Upgrading is recommended for anyone reading archives they did not produce.

  • FileEntry#getData() now throws the new ERR_ENTRY_DATA_OUT_OF_BOUNDS when the declared data of an entry, i.e. its offset plus its compressed size, ends past the end of the zip file. Such an entry used to make getData() hang for ever, with no error and no CPU use, so nothing timed out and nothing showed up in a profile. Honestly truncated archives are affected as much as malformed ones. A read past the end of the source now ends the stream instead of stalling it, which also covers the entries the bounds check cannot detect in advance
  • The output of FileEntry#getData() is no longer allocated from the declared uncompressed size of the entry. The size was reserved before a byte was read, so a 131-byte archive declaring 3 GiB reserved 3 GiB. The allocation is clamped to what the compressed data can decode to, compressedSize * 1032 for a compressed entry and compressedSize for a stored one, 1032 being the maximum expansion ratio of Deflate. The clamp never binds on real data, a legitimate archive still preallocates exactly its uncompressed size

New features

  • ZipFS, ZipEntry, ZipFileEntry and ZipDirectoryEntry are now exported at the top level, and the fs namespace is deprecated. Replace new zip.fs.FS() with new zip.ZipFS(), and zip.fs.ZipFileEntry with zip.ZipFileEntry. zip.fs keeps working and the library emits no runtime warning, the deprecation is documentation only. ZipEntry is now a value as well, so entry instanceof zip.ZipEntry works. The three entry classes were already declared as top-level exports but existed at runtime under zip.fs.* only, so importing them type-checked and then failed. In TypeScript, the FS type is deprecated and kept as an alias of ZipFS, so let fs: FS keeps compiling
  • New ZipEntry#setOptions() method and ZipEntry#options property in the filesystem API. setOptions() merges the options into the ones the entry was added with, an option set to undefined being removed instead of stored, and they are applied when the zip file is exported. It is the way to set the options of an entry imported from a zip file, which has none until it is called. The options describing the data of an entry exported with passThrough, e.g. compressionMethod and uncompressedSize, are ignored, they are always the ones of the original entry, and so are directory and the progress callbacks
  • ZipWriter#prependZip() now writes a correct split zip file when the writer is a split zip file writer. The whole prepended archive used to be copied into the first disk, so every entry recorded an offset on the wrong disk. The data is copied disk by disk now, a disk is closed before an entry whose local file header would not fit in what is left of it, and each entry records the disk it starts on and its offset in that disk. The output also starts with the split zip file signature, unless the prepended zip file already carries one
  • TextWriter now decodes CP437. new TextWriter("cp437") used to return the data decoded as UTF-8, since the encoding was handed to FileReader#readAsText(), which falls back to UTF-8 for a label it does not know. It goes through the same decoder as the filenames and the comments now. It also decodes with TextDecoder instead of FileReader, which removes the last dependency on that class, missing from some worker scopes. The byte order mark is still removed, whichever branch decodes the data
  • The second argument of the codec stream constructors is now typed, by the exported CompressionStreamOptions and DecompressionStreamOptions interfaces. They document which members are set for which class, e.g. deflate64 only for the deflate implementations, and rawBitFlag, compressionMethod and uncompressedSize only for the codecs registered with registerCodec(). Configuration#CompressionStream, Configuration#DecompressionStream, their *Fallback and deprecated *Zlib forms and CodecDefinition are declared with them instead of the untyped TransformStreamLike. This only concerns you if you pass a custom stream implementation or call registerCodec()
  • Configuration#baseURI is now declared. It resolves the relative workerURI, wasmURI and codecURI values, and defaults to the URL of the module of zip.js
  • WritableWriter#size is now declared. zip.js sets it to 0 before the first write and keeps it updated, so a custom Writer can read how many bytes have been written so far, e.g. to compute the offset of a disk. It is declared on Writer, TextWriter, BlobWriter, SplitDataWriter and Uint8ArrayWriter as well
  • Members that existed and were not declared: HttpReader#url, TextWriter#encoding, BlobWriter#contentType, Data64URIWriter#contentType, EntryError#overlappingEntry and EntryError#reason. overlappingEntry is the only way to identify the other entry of the pair reported by ERR_OVERLAPPING_ENTRY, and reason describes the ambiguity reported by ERR_AMBIGUOUS_ARCHIVE

Behavior changes

The options listed first used to accept values of the wrong type and produced a wrong, empty or silently dropped result. They throw now. If your code passes the documented types, nothing changes.

  • lastModDate, lastAccessDate and creationDate must be Date instances and throw the new ERR_INVALID_DATE otherwise. An invalid Date used to be written as an entry carrying no timestamp at all. A timestamp expressed in milliseconds is the natural mistake and is rejected as well: pass new Date(file.lastModified), not file.lastModified
  • The comment option of an entry must be a string and throws the new ERR_INVALID_ENTRY_COMMENT_TYPE otherwise. A Uint8Array used to be coerced and its textual representation written into the archive. Decode the bytes to a string before passing them
  • The extraField option must be a Map, and throws the new ERR_INVALID_EXTRAFIELD otherwise. Its keys must be integers between 0 and 65535, and ERR_INVALID_EXTRAFIELD_TYPE now covers a non-integer or a negative key as well as a key above 65535. Its values must be Uint8Array instances, and throw the new ERR_INVALID_EXTRAFIELD_DATA_TYPE otherwise
  • The readerOptions option of ZipDirectoryEntry#export*(), ZipDirectoryEntry#getExportedSize() and ZipDirectoryEntry#exportFileSystemHandle() must be an object and throws the new ERR_INVALID_READER_OPTIONS otherwise. A value of another type was silently ignored: a password passed as a string instead of an object failed with the unrelated ERR_ENCRYPTED, while the other options were dropped without any error. An unknown property of a readerOptions object is still ignored, as everywhere else in the API
  • The options expecting a function throw the new ERR_INVALID_FUNCTION_OPTION when they are given a value of another type: encodeText, decodeText, createTempStream, signCentralDirectory and decryptCentralDirectory. A falsy value keeps meaning "use the default"
  • The signal option throws the new ERR_INVALID_SIGNAL when it does not look like an AbortSignal, i.e. when it does not expose an addEventListener() method and a boolean aborted property. Duck-typed signals and signals coming from another realm keep working
  • The password and rawPassword options are now checked on the reader side as well, throwing ERR_INVALID_PASSWORD_TYPE. A value of another type used to fail with the unrelated ERR_ENCRYPTED or ERR_INVALID_PASSWORD
  • msdosAttributesRaw throws ERR_INVALID_MSDOS_ATTRIBUTES when the value is not an integer, and accepts a numeric string like the other numeric options. The range check used to be the only one, and the bitwise arithmetic folding the value into the external file attributes did the rest quietly: a fractional value was truncated, and a value that is not a number at all passed both comparisons and was written as 0. msdosAttributes throws ERR_INVALID_MSDOS_DATA on an array, which used to be accepted as an object and wrote 0 as well, since none of the flag properties exist on it
  • configure() and setDefaultConfiguration() reject two kinds of bad input instead of storing them. maxWorkers must be an integer greater than 0 and throws the new ERR_INVALID_MAX_WORKERS otherwise: a value lower than 1 used to deadlock ZipWriter#add() for ever, since no entry could start and none could release the next one. createWorker and the CompressionStream and DecompressionStream options, including their *Fallback and deprecated *Zlib forms, must be functions and throw ERR_INVALID_FUNCTION_OPTION otherwise, a falsy value still meaning "use the default". The numeric options accept a numeric string and are coerced, like the numeric options of the reader and the writer. Nothing is stored unless the whole call passes, so a rejected call leaves the configuration untouched
  • chunkSize is normalized wherever it is read. A value lower than 64 is raised to 64, as before, and a value that is not an integer greater than 0 now falls back to the default of 65536 instead of being used as it is. This applies to the global configuration and to the chunkSize option of Reader#createReadable()

The rest of this section changes results rather than rejecting input.

  • The configuration is read when it is used instead of when the reader or the writer is constructed. A configure() call made between new ZipWriter() and the first add(), or between new ZipReader() and the first getData(), used to be ignored and is honored now. It affects maxWorkers, chunkSize, the compression stream implementations and the deflate support detection
  • The executable option now counts as Unix metadata, like unixMode. It means a mode of 0o755, and it was the only Unix metadata option taking the MS-DOS branch, so an entry written with executable set next to msDosCompatible, msdosAttributes or msdosAttributesRaw lost its executable bit without a word. It wins over the three of them now, and selects the Unix platform for the "Version made by" field. executable set to false changes nothing, as before
  • The entries of the filesystem API are dated when they are added, not when they are written. An entry added without a lastModDate option used to be stamped with the current date at export time, so exporting an unchanged tree twice produced different bytes. The four sources of the date of an exported entry now rank as follows, weakest first: the moment the entry was added, the date of the entry the tree was imported from, the lastModDate option passed to the export, and the lastModDate option passed when the entry was added
  • The directories implied by the name of an imported entry are no longer written back when the tree is exported. Importing a zip file storing "a/b.txt" and no directory entry creates a navigable "a" entry, which used to be exported as an entry of its own, so a round trip gained one entry per path component. Only the directories carried by the source zip file and the ones created with addDirectory() are written now. getExportedSize() and the progress callbacks count them the same way
  • The options passed when an entry was added no longer override the values describing the data of an entry exported with passThrough. compressionMethod, uncompressedSize and the other pass-through values now win over the per-entry options, as they already did over the options passed to the export
  • The options passed when an entry is added are now copied. The object was stored as it was, so mutating it afterwards, or reusing one object for several entries and mutating it in between, changed entries that had already been added
  • The split zip file signature at the start of a zip file is no longer reported as prepended data. A single-disk archive written by SplitDataWriter starts with the four bytes PK\x07\x08, which used to be read as prepended data by extractPrependedData and rejected as an ambiguous archive by strictness set to "strict" or by checkAmbiguity. The temporary spanning marker PK00, which PKZIP writes at the start of the first disk while a spanned archive is being created, is accepted the same way. If you read archives produced by SplitDataWriter with either of those options, they are no longer misreported

Bug fixes

  • ZipWriter#prependZip() no longer copies the central directory of the zip file it prepends. The whole source was piped into the output, then the entries were rewritten after it, so every prepended archive carried its old central directory as dead bytes in the middle of the result. Only the data region is copied now, i.e. everything before the first byte of the central directory. Archives already produced this way are still readable, they are only larger than they need to be
  • ZipWriter#prependZip() reads the zip file it prepends once instead of twice. A reader providing readUint8Array() was buffered into a Blob because the central directory was read through reader.readable, so a BlobReader or an HttpRangeReader was fully downloaded into memory before anything was written
  • The transferred streams are no longer broken in a web worker installing a polyfill of the Streams API, the setup documented under Configuration#createWorker below. The worker received a native ReadableStream and a native WritableStream through postMessage() and piped them into streams of the polyfill, which rejected them. They go through the same compatibility wrappers as the streams created in the worker now

Documentation

  • msdosAttributesRaw and msdosAttributes now describe the platform they select. The behavior is unchanged: setting either of them selects the MS-DOS platform for the entry exactly as msDosCompatible set to true does, and overrides that option when it is explicitly set to false, so versionMadeBy loses its Unix upper byte and no Unix mode is written. What counts is that the option is provided, not its value, so 0 and {} select it too. Any Unix metadata option wins over the three of them, with the MS-DOS attributes written into the low byte, see the executable change above
  • ZipWriter#prependZip() now states that the data of the zip file is copied, its central directory rebuilt and its entries relocated, so the disks of a split zip file passed as input are unrelated to the disks of the output
  • ZipReader#comment and the comment option of an entry now explain why one is bytes and the other a string. The encoding of the comment of an entry is recorded in its header by the general purpose bit 11, the encoding of the global comment is recorded nowhere, so it can only be decoded with the encoding agreed with the producer of the zip file
  • Configuration#createWorker now documents how to install a polyfill of the Streams API in the scope of the worker, which is the way to run the web workers on the engines where TransformStream is missing from that scope, e.g. Firefox before version 102. A polyfill imported by the page does not help, because the worker reads the globals of the Streams API from its own scope
  • Configuration#workerURI now states that the worker is created as a module worker, unless the URI is a Data URI or a Blob URI, in which case it is created as a classic worker
  • ZipDirectoryEntryExportOptions now documents the precedence of the four sources of the last modification date of an exported entry, and ZipDirectoryEntry#importZip() documents that the directories implied by an entry name are not written back
  • File#lastModified is now a link to MDN in the generated documentation, instead of an unresolved reference

Tests and continuous integration

  • A new audit checks that every property name reaching the public API is either declared in index.d.ts or mangled on purpose. It reads the terser configuration and the declarations, walks the objects the library builds at runtime and reports the names belonging to neither list. It found eight internal fields shipped under their source names in the minified builds, e.g. the pending characters of Data64URIWriter and the source blob of BlobReader, which are mangled now, two of them renamed on the way. The parameter names of the declarations are no longer reserved either: a parameter name is not a property name, and reserving it kept a field of the same name readable in every build
  • The internal methods writing the records of a zip file are renamed away from the DOM property names they shared, so that they are mangled instead of being kept by the terser reserved list
  • Every public class is instantiated by the audit, so a member appearing only on an instance is covered. The classes reached only through a subclass, e.g. ZipEntry, are matched by walking the prototype chain
  • The deprecated checkSignature option is covered by a test of its own, and the rest of the suite uses checkCrc32. The option had lost all its usages when the tests were swept, so nothing exercised it any more
  • The zip.fs namespace is covered by a test of its own, for the same reason
  • The polyfill of the Streams API in the worker is tested with a classic worker, a module worker and the native build, one test per build since the WASM worker and the native worker bundle different codecs. The web runner gained a nativeBuild feature probe and caches the build probe instead of running it per test
  • New tests: the entries whose declared data extends past the end of the archive, the options of an entry of the filesystem API, the pass-through values against the per-entry options, the stability of the dates of an exported tree, the directories implied by an imported entry name, the encoding of TextWriter, the single read of prependZip(), the bytes it copies, its split zip file output, and the strictness of the split zip file signature
  • An option validation assertion is split into a test file of its own, since it needs the signal option of pipeTo(), which Chrome 76 to 79 ignore. The runner skips the file there rather than reporting a failure
  • The release is now scripted. npm run bump-patch bumps the version in package.json, package-lock.json and deno.json and commits it, the version script of npm syncing deno.json. A workflow step verifies that the three declared versions agree, and the release workflow verifies that the released tag matches the version declared in package.json

Credits

  • Andrew Chin, Brian J Lee and Youngjoon Kim, SSLab at Georgia Tech, reported the declared uncompressed size of an entry driving the allocation of the output
  • Claude (Opus 5) contributed to every change listed above

Full Changelog: https://github.com/gildas-lormeau/zip.js/compare/v2.8.53...v2.8.54

7 days ago
zip.js

v2.8.53

What's Changed in v2.8.53

New features

  • New checkLocalDirectory option in the reader options. It compares the local file header of an entry against its central directory record when FileEntry#getData() is called, and throws ERR_AMBIGUOUS_ARCHIVE when the two disagree. true compares the filename, the general purpose bit flag, the compression method, the CRC-32 checksum and the sizes, like strictness set to "strict"; false compares nothing, like "tolerant". Setting it explicitly always wins over strictness, whether strictness was passed to the constructor of ZipReader or to the call, so it is the way to ask for this one check without the archive-level checks of checkAmbiguity, and the way to drop it without giving up the other checks strictness performs. It is also the only way to validate the local file headers of a self-extracting archive, since checkAmbiguity rejects prepended data outright
  • The local file header of an entry now reports the two records the reader had already read and dropped. LocalDirectory#rawFilename holds the filename stored in the local file header, which is allowed to differ from EntryMetaData#rawFilename, and is defined when strictness is "strict" or checkLocalDirectory is true. LocalDirectory#dataDescriptor holds the data descriptor record written after the content, described by the new LocalDataDescriptor interface, and is defined when checkOverlappingEntry or checkOverlappingEntryOnly is set. It carries the CRC-32 checksum and the sizes stored in the record, each of which is allowed to differ from the central directory, and a signature flag telling whether the record is preceded by its optional signature. That signature is not part of the original format, it is a later convention writers are free to follow. When the four bytes look like the signature but the values behind them disagree with the central directory, the flag is false and the record is read as starting at those four bytes instead
  • The parsed extra field records are now typed instead of being declared as the bare EntryExtraField. EntryExtraFieldZip64, EntryExtraFieldNTFS, EntryExtraFieldExtendedTimestamp and EntryExtraFieldUnix describe the members the reader fills in, EntryExtraFieldUnicode gains version, filename and comment, and EntryExtraFieldAES gains compressionMethod, the real compression method of the entry, next to originalCompressionMethod, which is the 99 a WinZip AES header is required to carry in its place
  • SplitDataReader now accepts an array of Reader instances, of ReadableReader instances or of ReadableStream instances. The last two were declared in the TypeScript definitions and worked nowhere: reading a split archive requires the size of every disk to map a global offset onto one of them, so an element that only provides a stream is now buffered when the reader is initialized. This applies wherever an array of readers is accepted, i.e. the constructor of ZipReader, ZipWriter#add(), ZipWriter#prependZip(), ZipDirectoryEntry#importZip() and the reader property of a ZipFileEntry instance
  • ZipWriter#prependZip() now accepts a reader that only provides a ReadableStream. It reads the central directory of the archive it prepends before piping it, so passing a stream used to fail with TypeError: ReadableStream is already locked. The stream is buffered once, like the disks above
  • New ERR_INVALID_COMMENT_TYPE error constant

Behavior changes

  • The local file header of an entry is now compared against its central directory record by default, except for the filename. strictness set to "balanced", the default, used to trust the central directory record entirely; getData() now throws ERR_AMBIGUOUS_ARCHIVE when the general purpose bit flag, the compression method, the CRC-32 checksum or the sizes disagree. getEntries() is unaffected, the local file header is only read when the data is. This costs nothing: every one of those fields is read from the local file header anyway to locate the entry data. Only the filename is left out, because comparing it reads the filename bytes as well, which costs one extra read per entry whenever the local file header carries no extra field, the common case. "strict" still compares the filename too, "tolerant" still compares nothing, and checkLocalDirectory set to false restores the previous behavior. The new default was verified against 458,000 entries of real archives, where it rejects none of them
  • An explicit checkAmbiguity now wins over an inherited strictness. checkAmbiguity is the boolean form of strictness, true meaning "strict", and the two used to be resolved without regard to where they came from, so a checkAmbiguity passed to getEntries() or to getData() could not relax a strictness passed to the constructor of ZipReader. A value passed to the call now wins over a value passed to the constructor, and strictness still wins over checkAmbiguity when both are passed to the same one. checkAmbiguity set to false means "not strict" rather than "trust everything", so it downgrades an inherited "strict" to "balanced" and leaves an inherited "tolerant" alone; pass strictness set to "tolerant" to compare nothing. Code that passes strictness and never checkAmbiguity resolves exactly as before
  • The platform byte of the "Version made by" field is now forced instead of being merged into the value given by the versionMadeBy option. It is set to Unix (3) when the entry carries Unix metadata, i.e. when uid, gid, unixMode or unixExtraFieldType is set, and to MS-DOS (0) when msdosAttributes or msdosAttributesRaw is set. Only the lower byte of the given value survives in both cases. It used to be combined with the byte already present, so a versionMadeBy carrying another platform produced a value belonging to neither
  • ZipWriter#close() now throws the new ERR_INVALID_COMMENT_TYPE error when the comment it is given is not a Uint8Array. Passing a string, the natural mistake, used to fail deep inside the writer with TypeError: Cannot read properties of undefined (reading 'byteLength'), after the entries had been written. getExportedSize() performs the same check on the globalComment option

Bug fixes

  • The Unix user and group ids are now read from the local file header when the central directory has none. The Info-ZIP Unix type 2 extra field (0x7855) stores them in the local file header only and leaves a zero-length copy in the central directory, so uid and gid were undefined on every archive written by Info-ZIP. They are filled in when the data of the entry is read: they are still undefined after getEntries() and appear once getData() has run, since that is when the local file header is read, and they are also readable on EntryMetaData#localDirectory. A value read from the central directory is never overwritten by the local file header, since the type 2 field truncates the ids to 16 bits while the New Unix field (0x7875) does not
  • An empty Info-ZIP Unix type 2 extra field no longer hides the ids of the Info-ZIP New Unix extra field next to it. The reader looked at 0x7875 only when 0x7855 was absent, so an entry carrying both, which is what Info-ZIP writes, reported no ids at all although 0x7875 held them
  • EntryMetaData#rawLastAccessDate and EntryMetaData#rawCreationDate are now filled from the NTFS extra field. They were declared but never set: the raw FILETIME values were stored on the extra field record only. EntryMetaData#rawLastModDate is unaffected, it remains the MS-DOS date and time stored in the header
  • The entries returned by ZipReader#getEntries() now carry rawBitFlag, filenameLength, extraFieldLength and unixExternalUpper. The four properties were declared on EntryMetaData and read from the central directory, they were simply dropped when the entry object was built
  • unixExternalUpper is now the upper half of the externalFileAttributes the entry was written with, on the entry returned by ZipWriter#add(). It was computed before the unixMode option and the Unix file type were folded in, so it reported the default 0o644 for every entry, whatever the mode: an entry written with 0o120777 disagreed both with its own externalFileAttributes and with what the reader reports for it
  • A worker that fails to load now falls back to the main scope instead of throwing a TypeError. The codec pool builds the worker and its interface, then calls it back one turn later; when the error event of the worker arrived in that interval, the error was dropped and the pool went on to post a message to a worker it had already discarded, which failed with Cannot read properties of null (reading 'postMessage'). The designed fallback now runs in that case too, with the error of the worker as the reason. This affects the engines where a worker cannot be started at all, e.g. Firefox extensions using manifest v2 and Chromium 76 to 79
  • The entry returned by ZipWriter#add() now defines the same members as the entries returned by ZipReader#getEntries(): zip64, symlink, encrypted, zipCrypto and msDosCompatible were left undefined instead of false on one side or the other, and the deprecated internalFileAttribute and externalFileAttribute aliases were missing from it

Documentation

  • The strictness option now lists the fields each level compares, and states which of them are read from the local file header anyway
  • The versionMadeBy, msDosCompatible and unixMode options now describe how the platform byte and the Unix file type are chosen, including the fact that a folder entry is always written with S_IFDIR whatever type the mode carries
  • The symlink property now points at the option that writes a symbolic link, since there is no option of that name: the file type goes in unixMode, i.e. 0o120777 with the path of the target as the content of the entry
  • Several documented defaults disagreed with the code and were corrected: maxWorkers falls back to 2 when the environment provides no navigator.hardwareConcurrency, workerURI points at the worker of the build that was imported, the four CompressionStream and DecompressionStream options default to the global implementations or to the one embedded in the entry point, lastAccessDate and creationDate have no default at all so that the entries do not carry a meaningless time, and versionMadeBy defaults to 768 rather than 20
  • preventHeadRequest now states that leaving it unset is not the same as setting it to false when useRangeHeader or forceRangeRequests is set: the size is then read from a ranged GET request, and only an explicit false restores the HEAD request
  • rawLastModDate now states that it is the MS-DOS date and time of the header and is not replaced by the value of the NTFS extra field, unlike lastModDate

Tests and continuous integration

  • A workflow step rebuilds the project on every push and fails when the committed build output differs. The release workflow publishes the committed files as they are, it never rebuilds, so a stale dist/ or index.min.js at a tag would ship to npm and JSR. Pull requests are exempt, asking outside contributors to commit build output would add an unreviewable diff to every change
  • The release workflow now publishes only when the test suite passed on the released commit
  • A new audit compares the shapes of the objects the library builds at runtime against the interfaces declared in index.d.ts, in addition to the audit of the read and write surfaces added in v2.8.52. It is what found the members left undefined and the extra field records declared as the bare EntryExtraField
  • A regression test covers the backpressure of the writer on the web worker path
  • The browser runner restarts the browser and runs the suite again when the session is lost, and its --headful option was renamed to --headed
  • The Safari job runs on macOS 15 instead of macOS latest, where the browser loses its window or its session in the middle of the suite more often
  • The workers are terminated between the tests in the Node.js, Deno and Bun runners, and the Bun runner sets its own timeout
  • A test reproduces the ordering that made a failing worker throw instead of falling back, i.e. the error of the worker arriving before the pool sends it its first message

Credits

  • Claude (Opus 5) contributed to every change listed above

Full Changelog: https://github.com/gildas-lormeau/zip.js/compare/v2.8.52...v2.8.53

8 days ago
zip.js

v2.8.52

What's Changed in v2.8.52

New features

  • New getExportedSize() method on ZipDirectoryEntry and FS instances. It returns the exact size of the zip file the matching export*() call would write, without writing it. It takes the same options as the export*() methods, so the value it returns is the one the export produces. It is meant for the Content-Length header of a streamed download. It throws the new ERR_UNDETERMINED_SIZE error when the size cannot be known before writing, i.e. when an entry is compressed, when an entry has no known size, when signCentralDirectory is set, and when the bufferedWrite option lets the physical layout depend on the order in which the entries are written. Encryption does not prevent the prediction, its overhead is a fixed number of bytes
  • New onentryprogress option in the export*() methods of the filesystem API. It is called once per written entry with the number of entries written, the total number of entries, and the entry itself. It reports the entries whereas onprogress reports the bytes. It is called after the entry has been written. When bufferedWrite is enabled the entries are written concurrently, so it counts the entries written instead of giving the position of the entry in the zip file
  • New globalComment option in the export*() methods of the filesystem API. It sets the comment of the zip file. The options of these methods are applied to every entry, so setting comment there comments each entry instead of the archive, exactly as lastModDate there sets the date of each entry. The zip file comment therefore needed a name of its own
  • importZip() now accepts a ZipReader instance in addition to the data of a zip file. The caller builds the reader, passes it, and keeps it after the import. This is the way to read prependedData, appendedData, comment, digitalSignature, directoryOffset and directoryLength, which are only filled once the entries have been read. The options of the reader are merged with the options of the import, and the options of the import win
  • New symlink property on entries. It is true when the entry is a symbolic link. The target of the link is the content of the entry, which is read like any other entry, e.g. with getData(new TextWriter()). The target is not validated, it can be an absolute path or escape the directory of the entry, so it must be checked before being used. Writing a symbolic link is done by setting the unixMode option to a mode carrying the S_IFLNK type, e.g. 0o120777, with the path of the target as content
  • exportFileSystemHandle() now accepts the readerOptions option. The password option must be set there to export the entries of an encrypted zip file, since the password option of the export encrypts the written entries instead
  • The readerOptions option of the filesystem export now accepts passThrough. The entries imported from a zip file are then written as-is, without being decompressed and decrypted, exactly as importing them with this option does. The entries added to the filesystem are compressed as usual
  • The signCentralDirectory option is now declared in the export options of the filesystem API. It was already forwarded to the ZipWriter instance, it was simply missing from the TypeScript definitions
  • New ERR_UNDETERMINED_SIZE, ERR_INVALID_PASS_THROUGH and ERR_UNSUPPORTED_ENCRYPTION_PASS_THROUGH error constants

Behavior changes

  • The export*() methods of the filesystem API now report the progress of the whole archive instead of the progress of each entry. onstart and onend are called once, with the total size of the entries and with the number of bytes written. They used to be called once per entry, while onprogress was already reporting the archive as a whole, so the three callbacks disagreed with each other. onprogress is unchanged. Use the new onentryprogress option to be notified for each entry
  • The export options of the filesystem API now take precedence over the metadata of the entries imported from a zip file. Setting lastModDate, comment, versionMadeBy, uid, gid or the file attributes in the export options used to have no effect on those entries, although unixMode and msdosAttributes did reach them, so the same attribute word was writable one way and not the other. The order is now the metadata of the source entry, then the export options, then the description of the entries copied as-is, then the options of the entry. The description of the entries copied as-is stays above the export options because it describes the bytes being copied, not an intent. A level or a compressionMethod winning over it would write headers that do not match the content. An export using the default options is unchanged, so round trips keep their fidelity
  • ZipReader#close() now cancels the ReadableStream instance passed to the constructor when nothing has been read from it. It used to do nothing at all. The stream of a reader whose entries have been read is left alone, and so is a Reader instance, which belongs to the caller. The entries stay readable after the call
  • The preventClose option is now honored only when the caller owns the writable, i.e. when a WritableWriter instance is passed to exportZip() or exportWritable(). It is ignored by the other export*() methods of the filesystem API, whose Writer instance can only return its data once its writable is closed. Setting it there used to prevent the export from ever resolving
  • Writing an entry with the passThrough option and a password now throws ERR_UNSUPPORTED_ENCRYPTION_PASS_THROUGH. The data is copied as-is, so it cannot be encrypted. The password used to be ignored silently and the entry was written unencrypted although its header announced encryption
  • The passThrough option is now ignored for the entries with no content. add("dir/", undefined, { directory: true, passThrough: true }) used to throw ERR_UNDEFINED_READER, so a ZipWriter instance created with passThrough set to true could not write a single directory
  • Writing an entry with the unixMode option now stamps the type of the file in the external file attributes. zip.js used to write a mode with no type, e.g. 0o000644 where Info-ZIP writes 0o100644, so unzip -l listed the entry as ?rw-r--r--. A mode already carrying a type is left untouched, and the externalFileAttributes option still writes the value verbatim
  • The executable property of an entry is now false for symbolic links. The permission bits of a link are always 0o777, so the flag was meaningless on every link

Bug fixes

  • Reading a range of data no longer emits empty chunks. Reader#createReadable() enqueued an empty chunk at the end of every entry whose size is known, and one extra read was issued for an entry of unknown size. The data was correct, the stream simply contained a chunk of length 0
  • getBlob() and getData64URI() now honor the MIME type they are given. A filesystem entry holding a Blob instance returned it as-is, with the MIME type it was created with, and ignored the requested one
  • The filesystem API now runs the overlapping entry check when checkOverlappingEntryOnly is set in its reader options. The option means "run the check and stop before reading the content" in the core API. The filesystem API needs the content, so it used to drop the option and skip the check altogether. It is now mapped to checkOverlappingEntry, so the check runs and the content is still read
  • The reader options that withhold the content of an entry are now ignored by the filesystem API. checkPasswordOnly, checkOverlappingEntryOnly and preventClose used to be forwarded to the entries read from a zip file, which returned no data
  • Progress is now reported for the filesystem entries that were not imported from a zip file. Reading such an entry pipes the reader to the writer without going through a codec, so onprogress never fired. It is now reported for every entry
  • exportFileSystemHandle() now counts the bytes it writes. It used to report the compressed size of each entry against the uncompressed size of the archive, so the progress stopped around 1% of the total, and it reported nothing at all for the entries that were not imported from a zip file
  • addFileSystemHandle() now applies the entry options to the directories it creates. They were applied to the files only
  • The dates of the Info-ZIP unix extra field are now read as signed timestamps. A date before 1970 was read as a date in 2106
  • Writing an entry whose name ends with a slash no longer corrupts the unixMode option. The type of the file was combined with the type already present in the mode instead of replacing it, so 0o100644 became a socket and 0o120777 became an invalid type. Both were then read back as neither a directory nor a symbolic link
  • The deprecated externalFileAttribute and internalFileAttribute options work again. They were removed when they were renamed to externalFileAttributes and internalFileAttributes. They are back as deprecated aliases, and an option passed to add() now takes precedence over the option of the ZipWriter instance whatever the spelling of each
  • addText() now stores the size of the text in bytes instead of its number of UTF-16 code units. Only getExportedSize() read that value, so the size was under-reported for any text containing non-ASCII characters. The export itself was correct
  • The buffer reserved for the expansion of the deflate algorithm is now allocated only for the entries that are really compressed. A stored entry reserved the same margin as a deflated one

Performance

  • The export*() methods of the filesystem API now keep a running total instead of summing the progress of every entry on every tick. The cost was quadratic in the number of entries. Passing onprogress on an archive of 10,000 entries added 50% to the duration of the export, and now adds nothing measurable

Documentation

  • The useCompressionStream option now states that the native API is used for compression only when level is undefined or equal to 6. CompressionStream does not support compression levels, so any other value compresses the data with the embedded implementation. It also states that the data produced at a given level can vary between platforms, and that useCompressionStream must be set to false to get the same output everywhere
  • The uid and gid options now state which unix extra field carries them
  • ZipReader#close() and the createTempStream option now describe what they do. A temporary stream must be able to hold a whole entry, because the local header written before it holds the size and the CRC-32 of the entry. Its readable side is therefore consumed only once its writable side has been closed. A factory returning new TransformStream() deadlocks, whereas the default buffers everything

Tests and continuous integration

  • The browser test runner uses Selenium instead of Playwright. It accepts --exe-path, --url-search, --build, --headful and --help, and the test-webkit script was renamed to test-safari
  • A workflow runs the test suite on every push, including the native build
  • A workflow checks that the markdown documentation is up to date with index.d.ts
  • The API documentation is published to the GitHub Pages site on each release
  • The web runner accepts a maxParallelTests parameter. Chromium 87 loses the wake-up of the backpressure of a stream when 16 tests run in parallel, so its jobs cap the parallelism at 4
  • The tests that check the abort reason are skipped on the browsers that ignore the signal of pipeTo() instead of being reported as failures

Credits

  • @danny0838 contributed the switch to Selenium, the test workflow and the arguments of the browser runner (#673)
  • Claude (Opus 5) contributed to every change listed above

Full Changelog: https://github.com/gildas-lormeau/zip.js/compare/v2.8.51...v2.8.52

11 days ago
zip.js

v2.8.51

What's Changed in v2.8.51

New features

  • getChildren() returns the children of a directory as an array, and all its descendants when the recursive option is set to true. It is available on ZipDirectoryEntry and FS instances. The descendants are ordered level by level, like the result of readdir(path, { recursive: true }) in Node.js, which is also the order in which the entries are written by the export*() methods. Unlike the entries property of FS, the array excludes the root directory, leaves no empty slot for removed entries, and can start from any directory. It is a snapshot taken when the method is called, so the tree can be modified while the array is being iterated

Behavior changes

  • The export*() methods of the filesystem API now write the entries in the same order whatever the value of the bufferedWrite option. Setting it to false used to write each branch of the tree entirely before moving to the next one, whereas the default writes the entries level by level, so an archive exported with bufferedWrite set to false does not have the same entry order as in the previous versions. Only the order changes: the entries, their content and their metadata are identical, and a directory entry still precedes the entries it contains

Compatibility improvements

  • exportFileSystemHandle() called with concurrent set to true now reports the failure that stopped the export on browsers which do not support the reason argument of AbortController#abort(), e.g. Firefox 79 and Chromium 87. zip.js used to recognize the cancellation of the sibling entries by the reason it had passed to abort(). These browsers discard that reason and report a plain AbortError instead, so the cancellation was reported as the cause of the failure and the original error was demoted into entryErrors. The cancellation is now tracked by zip.js itself and never read back from the platform (see #669)
  • exportFileSystemHandle() now rejects with an error instead of rejecting with undefined when the export is aborted through the signal option on these browsers. The reason passed to AbortController#abort() is discarded by the platform and cannot be recovered, so a DOMException named AbortError is thrown in its place. Its message is exposed as the new ERR_ABORTED constant. Testing error.name == "AbortError" now identifies an aborted export on every supported platform, whereas these browsers used to report a plain Error when the export was aborted before it started and an AbortError when it was aborted while an entry was streaming

Documentation

  • The API documentation of exportFileSystemHandle() now states that an entry flagged as a symbolic link is written as a regular file whose content is the path of the link target, since the File System Access API cannot create symbolic links

Tests

  • The test verifying that the abort reason of the caller is forwarded is now skipped on browsers without support for AbortSignal#reason instead of being reported as a failure. It moved to its own file and covers aborting before the export as well as aborting while an entry is streaming

Credits

  • @danny0838 reported the failure on Firefox 79 and Chromium 87 and ran the test suite on these browsers

Full Changelog: https://github.com/gildas-lormeau/zip.js/compare/v2.8.50...v2.8.51