# event

type: ci
user: dan
comment:
  Fix the way parenthesis in MATCH expressions are handled by FTS if the tokenizer considers them to be token characters.


type: ci
user: drh
comment:
  Make sure the group_concat() function returns an empty string, not a NULL, 
  if it has at least one input row. Fix for ticket [55746f9e65f8587].


type: ci
user: drh
comment:
  Include sqlite3rtree.h in the tsrc/ pile of source files during
  target_source in the main.mk makefile.


type: ci
user: mistachkin
comment:
  When cross-compiling with the MSVC makefile, make sure the correct library path is used.  Also, keep track of the required compilation options separately.


type: ci
user: drh
comment:
  Bring the PRAGMA logic into closer alignment with trunk, and in the process
  fix a couple of crashes.


type: ci
user: drh
comment:
  Initial attempt to merge in all trunk changes over the previous 1.5 years.
  This check-in compiles but there are compiler warnings and "make test"
  segfaults after only running a few test modules.


type: ci
user: drh
comment:
  Fix compiler warnings.  When forcing the delete of a WAL file, do not
  fail if the WAL files does not exist.  All "veryquick.tcl" tests are now
  passing on Linux under the standard compile-time options.


type: ci
user: drh
comment:
  In the sorter, only use large memory allocations if scratch memory has not
  been configured.  Add #ifdefs to disable unused code when 
  SQLITE_MAX_WORKER_THREADS is zero.  Other sorter changes in support
  of testability.


type: ci
user: mistachkin
comment:
  Merge updates from trunk.  Emit the AppliesTo and DependsOn SDK manifest attributes when building the VSIX for Windows Phone 8.1.


type: ci
user: mistachkin
comment:
  When using Visual Studio 2013, add the appropriate MaxPlatformVersion attribute to the VSIX SDK manifest.


type: ci
user: mistachkin
comment:
  In mutex_w32.c, make inclusion of the Windows header file dependent on SQLITE_OS_WIN, not SQLITE_MUTEX_W32.


type: ci
user: dan
comment:
  Add extra test to backcompat.test to ensure that old and new versions of FTS may work together on the same incremental merge operation.


type: ci
user: drh
comment:
  Repurpose the SQLITE_TESTCTRL_FAULT_INSTALL test-control to register a
  callback to be invoked by sqlite3FaultSim().  That test-control has been
  unused since 2008-06-20 and was never used in any official release.


type: ci
user: dan
ecomment:
  Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated. Allow the user to configure the number of segments merged simultaneously by the automerge option.

comment:
  Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated.


type: ci
user: dan
comment:
  Do not search for promotable segments following an FTS 'optimize' operation or creation of a new segment on the oldest existing level.


type: ci
user: dan
comment:
  Fix a bug causing an incorrect segment size value to be stored if both an old and new FTS version performed work on the same incremental merge operation.


type: ci
user: dan
comment:
  Fixes to prevent an FTS index from growing indefinitely as the corresponding table is updated. Change the FTS 'automerge' option to allow the user to specify the number of segments that should be merged simultaneously by auto-merges.


type: ci
user: drh
comment:
  Add a pair of sqlite3FaultSim(100) calls to vdbesort.c to facilitate testing
  of obscure and hard to reach error conditions.


type: ci
user: drh
comment:
  Changes to help ensure that a multiplication does not overflow when computing
  the number of bytes needed for a memory allocation, and cause a malfunction.
  No problems existing problems were discovered.  However, these changes should
  help to ensure that no problems arise in the future.


type: ci
user: drh
comment:
  In vdbesort.c, change the names of PmaReader variables "pIter" to "pReadr".
  Other related comment changes.  The generated object code should be the same.


type: ci
user: drh
comment:
  Fix the sqlite3_db_readonly() API so that it reports true if the database
  is readonly because the file format write version (byte 18 of the header)
  is too large.


type: ci
user: drh
ecomment:
  Make sure sufficient VDBE registers are allocated for an INSERT...SELECT
  when there is an idlist on the insert table that includes an explicit
  rowid.  Ticket [e9654505cfda93610585fde5a9bbf2e730c8a8d5]

comment:
  Make sure sufficient VDBE registers are allocated for an INSERT...SELECT
  when there is an idlist on the insert table that includes an explicit
  rowid.  Ticket [9654505cfda93610585fde5a9bbf2e730c8a8d5]


type: ci
user: drh
comment:
  In the command-line shell, if three or more interrupt signals (control-c)
  are received in a row without a response from sqlite3_interrupt() then
  call exit(1) immediately.  This allows control-C to interrupt the shell
  even if it is stuck in a computation or loop that does not involve the
  VDBE.


type: t
user: drh
brief: Fixed ticket [e9654505cfda9361|e9654505cf].
comment:
  Fixed ticket [e9654505cfda9361|e9654505cf]: <i>Inifinite loop in INSERT statement.</i> plus 5 other changes


type: ci
user: dan
comment:
  Changes to the output of EXPLAIN QUERY PLAN to make it clearer when a query uses the PRIMARY KEY index of a WITHOUT ROWID table.


type: ci
user: drh
comment:
  Remove references in comments to VDBE opcodes that no longer exist.  This
  is a documentation change only; no changes to code.


type: ci
user: dan
comment:
  Fix a problem in FTS4 where columns with names that are prefixes of any notindexed column were also being (incorrectly) marked as not indexed. For example in "CREATE ... t1(abc, bc, abcd, notindexed=abcd)", both abc and abcd were being treated as notindexed.


type: ci
user: dan
comment:
  Fix a problem in the shell when importing CSV files. If the leftmost field of the first row in the CSV file was both zero bytes in size and unquoted, no data was imported.


type: ci
user: dan
comment:
  Enable the OR optimization for WITHOUT ROWID tables. Use a temp table instead of the RowSet object to track the rows that have already been included in the result set.


type: ci
user: dan
comment: Add an extra test to further verify that the FTS notindexed option is working properly.
ecomment:
  Moved to "mistake" because this commit contains a typo causing a test to fail. Was: Add an extra test to further verify that the FTS notindexed option is working properly.


type: ci
user: drh
comment:
  Change the name of the Index.autoIndex field to Index.idxType and provide
  symbolic names for the various values of that field rather than using
  magic numbers.


type: ci
user: mistachkin
comment:
  Revise locking retry semantics in Win32 VFS to abort early if the file handle is reported as invalid.


type: g
user: dan
comment:
  Edit [7d445e593a99665a|7d445e593a]: Move to branch [/timeline?r=mistake&nd&dp=7d445e593a99665a&unhide | mistake]. Add propagating "hidden". Edit check-in comment.


type: ci
user: drh
comment:
  Improvements to error messages for errors in the dot-commands of the
  command-line shell.  Add the ".once" command.  The output of ".help"
  now goes to the designated output channel.


type: ci
user: drh
comment:
  Enhance the unix VFS so that it keeps track of the size of unlinked files
  internally and thus avoids the need to call fstat() on those files, since
  fstat() does not work reliably on unlinked files on some implementations of
  FuseFS.


type: ci
user: drh
comment:
  Fix the query planner so that it once again knows that queries without a
  FROM clause will never return more than one row and hence do not require
  sorting.


type: ci
user: drh
comment:
  Avoid closing a pseudo-table associated with sorting in a co-routine since
  that pseudo-table might be used again in an outer loop.


type: ci
user: dan
comment:
  Fix a problem that was causing the FTS4 integrity-check command to fail if the table had one or more "notindexed" columns.


type: ci
user: drh
comment:
  Make sure the estLog() routine in the query planner handles negative
  values correctly (always returning 1).  Prior to this change, estLog(-1)
  would return a large number which could throw off the cost computations
  in the query planner in obscure circumstances.


type: ci
user: drh
comment:
  Modify the %nonassoc directive in lemon so that it generates a run-time
  error rather than a parsing conflict.  This changes is due to a bug report
  on the mailing list.  SQLite does not use the %nonassoc directive in its
  grammar so this change does not affect SQLite.


type: ci
user: drh
comment:
  Proposed change to the fix for ticket [1c69be2dafc28b] such that legacy
  applications that were exploiting the older buggy behavior in SQLite
  continue to work.


type: ci
user: drh
comment:
  Enhance the unix VFS so that it keeps track of the size of unlinked files
  internally and thus avoids the need to call fstat() on those files, since
  fstat() does not work reliably on unlinked files on some implementations
  of FuseFS.


type: t
user: drh
brief: Open ticket [1c69be2dafc28b9f|1c69be2daf].
comment:
  Open ticket [1c69be2dafc28b9f|1c69be2daf]: <i>Incorrect GROUP BY when input and output columns have the same name</i> plus 4 other changes


type: ci
user: drh
comment:
  Fix the extension01.test script so that it works on windows.  Fix a 
  harmless compiler warning in the fileio extension.


type: ci
user: drh
comment:
  Add the ".eqp" dot-command to the ".help" output in the command-line shell.
  This was accidently omitted when the ".eqp" command was originally implemented
  for SQLite 3.8.4 (check-in [e6ecf7337658624]).


type: ci
user: drh
comment:
  Back out the unix VFS changes that seeks to avoid fstat() calls after an
  unlink().  That change did not clear the problem on FuseFS.


type: ci
user: drh
comment:
  Add the SQLITE_UNLINK_AFTER_CLOSE compile-time option.  If enabled, the
  unlink of temporary files on unix is deferred until after the file is closed.


type: ci
user: drh
comment:
  In the "wheretrace" logic (disabled for normal builds) expand the print width
  of the flags field to be wide enough for all 17 bits.


type: ci
user: mistachkin
comment:
  Fix CSV import issue, reported via the mailing list, in the shell when the file to be imported ends with an empty line.


type: ci
user: drh
comment:
  Allow an automatic index on a WHERE constraint where the RHS is does not
  reference another table as long as the RHS is not a constant.


type: ci
user: drh
comment:
  Simplification to the logic that prevents automatic indexes from being formed
  on constant expressions.


type: ci
user: drh
comment:
  A different approach to the autoindex problem that deletes query-planner code
  rather than adding it.


type: ci
user: drh
comment:
  Add the likely() function for symmetry with unlikely().  The likely(X)
  function means the same thing as likelihood(X,0.9375).


type: ci
user: drh
comment:
  Improvements to query planning, especially in regards to estimating the
  cost and benefit of automatic indexes.


type: ci
user: drh
comment:
  Add the ability to decode the headers of individual cells, byte-by-byte,
  in the "showdb.exe" utility.


type: ci
user: drh
comment:
  In the showdb.exe utility, for the "NNNbd" command, also show data fields
  in addition to header fields on each record decoded.  Improvements to 
  formatting.


type: ci
user: drh
comment:
  Add the showstat4.exe utility program for decoding and displaying the 
  content of the sqlite_stat4 table in a database.


type: ci
user: drh
comment:
  Add the .fullschema command to the sqlite3.exe utility.  This command shows
  the schema and the content of the sqlite_stat tables, all in one go.  Useful
  when reporting problems with the query planner.


type: ci
user: dan
comment:
  Attempt to use sqlite_stat4 data to estimate the number of rows visited by a range query that uses a skip-scan. This code is largely untested.


type: ci
user: dan
comment:
  Change the position list format so that its size in bytes is stored at the start of the list itself.


type: ci
user: dan
comment:
  Fix a couple of problems in estimating the number of rows visited by a range query that uses a skip-scan.


type: ci
user: drh
comment:
  Add makefile targets for various diagnostic tools, such as showstat4.
  Fix harmless compiler warnings in diagnostic tools.


type: t
user: drh
brief: Fixed ticket [9a8b09f8e69d3d9d|9a8b09f8e6].
comment:
  Fixed ticket [9a8b09f8e69d3d9d|9a8b09f8e6]: <i>Affinity problem on &quot;x IN (?)&quot;</i> plus 5 other changes


type: ci
user: drh
comment:
  Change the VDBE to export the sqlite3MemCompare() routine and thus free
  where.c from the dependency on vdbeInt.h.


type: ci
user: dan
comment:
  Fix a problem with using stat4 samples of type text when estimating the rows visited by a range-query/skip-scan loop.


type: ci
user: drh
comment:
  Add header comments on new routines.  Rework the sqlite3Stat4Column() routine
  so that is (in theory) able to deal with corrupt samples.


type: ci
user: drh
comment:
  Omit non-primary-key columns from the STAT4 samples for WITHOUT ROWID tables.
  Indexes, both rowid and without-rowid, still hold an exact copy
  of the index entry.


type: ci
user: dan
comment:
  Fix where.c so that the primary key values appended to every index entry on a WITHOUT ROWID table may be used when useful.


type: ci
user: drh
comment:
  Attempt to use STAT4 information to estimate the selectivity of WHERE clause
  terms when using the skip-scan optimization.


type: ci
user: drh
comment:
  Fix for ticket [b2fa5424e6fcb15]: Better define the format of the sqlite_stat4
  file for WITHOUT ROWID tables and make sure the ANALYZE command generates a
  file in the appropriate format.  Use the sqlite_stat4 data to enable the use
  of WHERE terms that cover all indexed columns plus some prefix of columns
  in the primary key.


type: t
user: drh
brief: Fixed ticket [b2fa5424e6fcb15b|b2fa5424e6].
comment:
  Fixed ticket [b2fa5424e6fcb15b|b2fa5424e6]: <i>Incorrect STAT4 samples for WITHOUT ROWID tables</i> plus 5 other changes


type: ci
user: dan
comment:
  Ensure that all fields are loaded from the stat4 table for records that correspond to indexes on WITHOUT ROWID tables with composite primary keys.


type: ci
user: dan
comment:
  Add another test to verify that SQLite is using stat4 data for composite primary keys on WITHOUT ROWID tables.


type: ci
user: dan
comment:
  Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE.


type: ci
user: drh
comment:
  Fix a division-by-zero error in the fts3view utility program.  Add the
  fts3view utility program to the "main.mk" makefile.


type: ci
user: drh
comment:
  SQLite has long accepted some unadvertised and non-standard join syntax.
  Add a test to ensure that future versions continue to accept this non-standard
  syntax, to avoid breaking legacy applications that use the undefined syntax.


type: ci
user: dan
comment:
  Update the sqlite3_stmt_busy() function so that it correctly returns true for "ROLLBACK" statements that have been stepped but not yet reset.


type: ci
user: drh
comment:
  Fix harmless compiler warnings in the showdb and showwal tools and in
  the unicode tokenizer of FTS3.


type: ci
user: mistachkin
comment:
  Add new ASCII mode to the shell capable of importing and exporting using the official unit and record separators (i.e. 0x1F and 0x1E, respectively).


type: ci
user: drh
comment:
  Limit the maximum SQLITE_MAX_ATTACHED to 127, since a signed character
  is sometimes used to store the database index.


type: ci
user: drh
comment:
  Allow the SQLITE_MAX_ATTACHED compile-time option to be larger than 62.  The
  default limit on the number of attached databases remains 10.


type: ci
user: drh
comment:
  Enhance the sqlite_stat1.stat parsing to allow additional text parameters
  at the end.  Unrecognized parameters are silently ignored.


type: ci
user: drh
comment:
  Correction:  The maximum SQLITE_MAX_ATTACHED value to avoid overflowing a
  signed 8-bit integer is 125, not 127.


type: ci
user: drh
comment:
  Fix the index name for the shadow tables in the spellfix1 extension so that
  multiple instances of the spellfix1 virtual table can each have their own
  index.


type: ci
user: drh
comment:
  The optimization of check-in [b67a6e33f2] does not work (it generates
  incorrect VDBE code) if an OR term is AND-ed with a constant expression.
  So back that optimization out and add a test case to make sure it does not
  get added back in.


type: ci
user: drh
comment:
  Add the OP_ReopenIdx opcode that works like OP_OpenRead except that it becomes
  a no-op if the cursor is already open on the same index.  Update the
  OR-optimization logic to make use of OP_ReopenIdx in order to avoid
  unnecessary cursor open requests sent to the B-Tree layer.


type: ci
user: drh
comment:
  When running ANALYZE, it is not necessary to check the right-most key column
  for changes since that column will always change if none of the previous
  columns have.


type: ci
user: drh
comment:
  Change the hex literal processing so that only the SQL parser understands
  hex literals.  Casting and coercing string literals into numeric values does
  not understand hexadecimal integers.  This preserves backwards compatibility.
  Also:  Throw an error on any hex literal that is too big to fit into 64 bits.


type: ci
user: drh
comment:
  Remove a surplus function prototype.  #ifdef code that is not used when
  hex integers are omitted at compile time.


type: ci
user: drh
comment:
  Improve the performance of the ANALYZE command by taking advantage of the
  fact that every row of a UNIQUE index is distinct.


type: ci
user: drh
comment:
  Ugh.  Consecutive UNIQUE index entries are only distinct if the index is
  on NOT NULL columns.  So the previous version was not quite right.  This
  check-in fixes the problem.


type: g
user: drh
comment:
  Edit [3e1e79e1335f7ad3|3e1e79e133]: Move to branch [/timeline?r=faster-analyze&nd&dp=3e1e79e1335f7ad3&unhide | faster-analyze].


type: ci
user: drh
comment:
  Avoid change tests when analyzing single-column unique indexes after
  getting past the initial NULL entries.


type: ci
user: drh
comment:
  In the command-line shell, in CSV output mode, terminate rows with CRNL but
  do not expand NL characters in data into CRNL.  Provide the extra -newline
  command-line option and the extra argument to .separator to designate an
  alternative newline character sequence for CSV output.


type: ci
user: drh
comment:
  Fix a bug in the whereRangeSkipScanEst() procedure (added by
  check-in [d09ca6d5efad3e4cfa]) where it fails to consider the possibility
  of a ROWID column when computing the affinity of a table column.


type: ci
user: drh
comment:
  Improve the performance of the ANALYZE command by taking advantage of
  UNIQUE constraints on indices.


type: ci
user: dan
comment:
  Add extension apis xRowCount, xQueryPhrase, xSetAuxdata and xGetAuxdata. And a ranking function that uses all of the above.


type: ci
user: drh
comment:
  Improvements to comments in the multi-threaded sorter.  Also include a
  function name change for clarity.  And add a test to help show that the
  MergeEngine object is only used by a single thread.


type: ci
user: drh
comment:
  In vdbesort.c, rename all pointers to sqlite3_file objects "pFd" and use the
  name "pFile" only for pointers to SortFile objects.  Other comment enhancements.


type: ci
user: drh
comment:
  In vdbesort.c, rename vdbeSorterDoCompare() to vdbeMergeEngineCompare() and
  move it closer to the one place where it is called.  Other minor comment
  changes.


type: ci
user: drh
comment:
  Remove an unnecessary parameter from vdbeMergeEngineStep().  Rename a couple
  other routines to be more descriptive of what they do.


type: t
user: dan
brief: New ticket [d2889096e7bdeac6|d2889096e7].
comment:
  New ticket [d2889096e7bdeac6|d2889096e7] <i>Problem with rtree tables with IN() operators or as the inner loop of a join.</i>.


type: t
user: dan
brief: Ticket [d2889096e7bdeac6|d2889096e7]: 6 changes
comment:
  Ticket [d2889096e7bdeac6|d2889096e7] <i>Problem with rtree tables with IN() operators or as the inner loop of a join.</i> status still Open with 6 other changes


type: ci
user: dan
comment:
  Have calls to the xFilter() method of rtree virtual tables ensure that cursor is initialized before proceeding. Fix for [d2889096e7bdeac].


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent trunk changes, and especially the fix for the R-Tree problem
  described in ticket [d2889096e7bdeac6].


type: t
user: drh
brief: Fixed ticket [d2889096e7bdeac6|d2889096e7].
comment:
  Fixed ticket [d2889096e7bdeac6|d2889096e7]: <i>Problem with rtree tables with IN() operators or as the inner loop of a join.</i> plus 3 other changes


type: ci
user: drh
comment:
  Ensure that the correct number of columns in a UNIQUE index are checked for
  uniqueness, regardless of whether or not the original table has a ROWID or
  if the columns are NOT NULL, etc.  Ticket [9a6daf340df99ba93c].


type: ci
user: drh
comment:
  Merge recent trunk changes, and especially the fix for the CREATE UNIQUE INDEX
  problem of ticket [9a6daf340df99ba9].


type: ci
user: drh
comment:
  Merge all recent trunk changes, and especially the R-Tree IN operator fix
  of ticket [d2889096e7bdeac].


type: ci
user: drh
comment:
  Add the "eForce" parameter to the sqlite3_multiplex_shutdown() entry point
  in test_multiplex.c.  Shutdown is forced if true.  Shutdown is not done if
  there are pending database connections and eForce is false, but an error log
  entry is made instead.


type: t
user: drh
brief: Fixed ticket [9a6daf340df99ba9|9a6daf340d].
comment:
  Fixed ticket [9a6daf340df99ba9|9a6daf340d]: <i>UNIQUE not enforced on an index</i> plus 5 other changes


type: ci
user: drh
comment:
  Add three new static mutexes for use by the application.  This is a partial
  import of changes from the threads branch.


type: ci
user: drh
comment:
  Add a new sqlite3FaultSim() call to vdbePmaReaderSeek() to facilitate
  tests of error handling in the sorter.


type: ci
user: drh
comment:
  Deactivate the DISTINCT in a SELECT on the right-hand side of an IN operator,
  since it should not make any difference in the output but dues consume extra
  memory and CPU time.


type: ci
user: drh
comment:
  The idea of coding IN operator with a short list on the RHS as an OR expression
  turns out to be helpful.  If the list is of length 1 or 2, the OR expression
  is very slightly faster, but the ephemeral table approach is clearly better for
  all list lengths greater than 2.  Better to keep the code simple.


type: ci
user: dan
comment:
  Add "doclist index" records to the database. These are to make navigating within very large doclists faster. They are not yet used by queries.


type: ci
user: drh
comment:
  Omit a pointless OP_Null when processing a value-list RHS of an IN operator
  where the LHS is a rowid.


type: ci
user: drh
comment:
  Refactoring:  Change "pIndex->onError!=OE_None" to use a macro:
  "IsUniqueIndex(pIndex)".  Easier to understand that way.


type: ci
user: drh
comment:
  Begin making changes to the IN operator in an attempt to make it run faster
  and to make the code easier to understand.


type: ci
user: dan
comment:
  Have the fts5 integrity-check verify that doclist indexes match the contents of the leaf pages that they index.


type: ci
user: drh
comment:
  Rename the IsNull opcode to IfNull and rename the NotNull opcode to
  IfNotNull.

ecomment:
  Changed my mind:  This opcode name changes mere creates unnecessary diff marks
  between older and newer versions without significantly improving readability.
  Was: Rename the IsNull opcode to IfNull and rename the NotNull opcode to
  IfNotNull.


type: ci
user: drh
comment:
  Remove an unnecessary OP_Null in the IN-operator logic.  Attempt to clarify
  comments explaining the IN-operator code, though it is not clear that the
  comments are correct even yet - more work to be done.


type: ci
user: drh
comment:
  Clean up the IN operator code generation logic to make it easier to reason
  about.  In the process, improve code generation to omit some unused OP_Null
  operations.


type: ci
user: dan
comment:
  Add a special case to the integrity-check code to check that the final integer in a doclist index is as expected.


type: ci
user: drh
comment:
  Enhancements to the code generator for the IN operator that result in much
  faster queries in some cases, for example when the RHS of the IN operator
  changes for each row of a large table scan.


type: g
user: drh
comment:
  Edit [2f724cbac9025cea|2f724cbac9]: Move to branch [/timeline?r=deadend&nd&dp=2f724cbac9025cea&unhide | deadend]. Mark "Closed". Edit check-in comment.


type: ci
user: drh
comment:
  Part of the change in the previous check-in was incorrect and can result
  in an incorrect UPDATE for WITHOUT ROWID tables.  This check-in fixes the
  problem.


type: ci
user: drh
comment:
  Add the ability to evaluate IN operators as a sequence of comparisons as
  an alternative to the long-standing algorithm of building a lookup table.
  Use the new implementation in circumstances where it is likely to be faster,
  such as when the RHS of the IN changes between successive evaluations.


type: ci
user: drh
comment:
  Ensure that aggregate functions are not used when evaluating a default
  value for a table column.
  Candidate fix for ticket [3a88d85f36704eebe134f7].


type: t
user: drh
brief: New ticket [3a88d85f36704eeb|3a88d85f36].
comment:
  New ticket [3a88d85f36704eeb|3a88d85f36] <i>Segfault when a DEFAULT value contains an aggregate function</i>.


type: ci
user: drh
comment:
  A simpler fix for ticket [3a88d85f36704eebe1] - one that uses less code.
  The error message is not quite as good, but as this error has apparently
  not previously occurred in over 8 years of heavy use, that is not seen as
  a serious problem.


type: t
user: drh
brief: Fixed ticket [3a88d85f36704eeb|3a88d85f36].
comment:
  Fixed ticket [3a88d85f36704eeb|3a88d85f36]: <i>Segfault when a DEFAULT value contains an aggregate function</i> plus 5 other changes


type: ci
user: mistachkin
comment:
  In the Win32 VFS, work around InterlockedCompareExchange() being a macro on some platforms (e.g. x64).


type: ci
user: drh
comment:
  On the windows VFS, do not try to make InterlockedCompareExchange an
  overloadable function, since sometimes it is a macro.


type: ci
user: drh
comment:
  Fix harmless compiler warnings.  In the command-line shell, report if the
  ".system" command returns a non-zero result.


type: ci
user: drh
comment:
  Fix two more harmless compiler warnings.  Make sure the fts3_unicode2.c file
  is in sync with mkunicode.tcl.


type: ci
user: drh
ecomment:
  Oops!  This check-in was on trunk.  But it contains a debugging printf().
  Original comment:
  When the estimated cost to do a sort overwhelms the estimated cost to do
  individual table lookups, make sure that the table lookup costs are still
  taken into consideration when selecting the lookup algorithm.

comment:
  When the estimated cost to do a sort overwhelms the estimated cost to do
  individual table lookups, make sure that the table lookup costs are still
  taken into consideration when selecting the lookup algorithm.


type: ci
user: dan
comment:
  Add "segment promotion" to fts5. This prevents the FTS index from growing indefinitely as data is added and deleted.


type: ci
user: dan
comment:
  Use multiple memory allocations for a single Fts5Structure object. This is probably less efficient but much easier to get right.


type: g
user: drh
comment:
  Edit [ec5d84ba69c100d9|ec5d84ba69]: Move to branch [/timeline?r=query-planner-fix&nd&dp=ec5d84ba69c100d9&unhide | query-planner-fix]. Edit check-in comment.


type: ci
user: drh
comment:
  Reworking the documentation on integer result codes.  This is a comment
  and documentation change only.  There are no changes to code.


type: ci
user: drh
comment:
  When the estimated sorting cost overwhelms the estimated lookup cost, ensure
  that lookup costs are still taken into account when selecting a lookup
  algorithm.


type: ci
user: drh
comment:
  The SQLITE_IOERR_BLOCKED extended error code is not longer used, so remove
  assert() statements and documentation for that error code.  Also make other
  documentation improvements.


type: ci
user: dan
ecomment:
  Because SQLite internally calculates query plan costs using a logarithmic scale, very large estimated sorting costs can cause all other estimated costs to be rounded down to zero. In these cases break ties between plans with the same total cost by comparing the costs with sorting excluded. This is an alternative fix for the problem addressed by [2af630c572].

comment:
  Because SQLite internally calculates query plan costs using a logarithmic scale, very large estimated sorting costs can cause all other estimated costs to be rounded down to zero. In these cases break ties between plans with the same total cost by comparing the costs with sorting excluded. This is an alternative fix for the same problem as addressed by [2af630c572].


type: ci
user: drh
comment:
  Improvements to the way the query planner handles sorting costs, so that
  very large sorting costs do not overwhelm the loop costs.


type: ci
user: drh
comment:
  Clarify the comment explaining the meaning of the SQLITE_WIN32_GETVERSIONEX
  macro in the Windows VFS.


type: ci
user: dan
comment:
  Run a test with TEST_REALLOC_STRESS and OMIT_LOOKASIDE defined as part of releasetest.tcl on Linux/x86-64.


type: ci
user: dan
comment:
  If SQLITE_TEST_REALLOC_STRESS is defined, extend the op-code array used by virtual-machine programs by one element at a time, instead of doubling its size with each realloc().


type: ci
user: drh
comment:
  Improve the comments associated with SQLITE_TEST_REALLOC_STRESS and add
  an extra assert() to prove an assumption.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  A reasonably complete implementation of the "changeset" command-line tool
  and the ".sessions" command in the command-line shell.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Begin adding the sqlite3session_patchset() API to the sessions extension. This is an interim commit.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge the latest trunk changes, and in particular the refactoring of the
  object names in the command-line shell.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Begin adding commands to the command-line interface for interacting with
  the sessions extension.  This is the first check-in of a work-in-progress.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Add the "changeset" command-line tool for analyzing and manipulating
  changesets in files on disk.  Add the ".session" command to the command-line
  tool.


type: ci
user: dan
comment:
  Add documentation for tokenizer api to fts5.h. Also add a script to extract extension API docs and format them as html.


type: ci
user: drh
comment:
  Change an assert() added by the previous commit into a testcase().
  Fix a separate assert() in btree.c:freeSpace().


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Fix the autoconf and MSVC makefiles so that they construct the changeset
  command-line utility upon request.  Delete that utility program when
  "make clean" is run.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Improved response to error conditions in the ".session" shell command and
  in the "changeset" command-line program.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge all recent trunk changes, especially the fix for
  ticket [369d57fb8e5ccdff06f1], but also the skip-scan improvement and
  performance improvements in the b-tree code.


type: ci
user: drh
comment:
  Make sure the sqlite3_trace() callback is invoked, even if the prepared
  statement was marked "expired" before it ever entered sqlite3_step().
  Ticket [11d5aa455e0d98f3c1e6a08].


type: t
user: drh
brief: New ticket [11d5aa455e0d98f3|11d5aa455e].
comment:
  New ticket [11d5aa455e0d98f3|11d5aa455e] <i>sqlite3_trace() sometimes does not invoke its callback</i>.


type: t
user: drh
brief: Fixed ticket [11d5aa455e0d98f3|11d5aa455e].
comment:
  Fixed ticket [11d5aa455e0d98f3|11d5aa455e]: <i>sqlite3_trace() sometimes does not invoke its callback</i> plus 5 other changes


type: ci
user: drh
comment:
  Modify the memsubsys1-3.1.4 test so that it does not fail arbitrarily due
  to variations in the behavior of system malloc().


type: g
user: drh
comment:
  Edit [1cb1cd64930a6bc3|1cb1cd6493]: Move to branch [/timeline?r=btree-speedup&nd&dp=1cb1cd64930a6bc3&unhide | btree-speedup].


type: ci
user: drh
comment:
  Size reduction and performance improvements in btree.c and the
  allocateSpace() routine.  Also fix an assert() in freeSpace().


type: ci
user: drh
comment:
  A small performance improvement in freeSpace() by special-casing the
  relatively common case of an empty freelist.


type: ci
user: drh
comment:
  Enhancements to skip-scan such that it is operable when a middle column of
  an index is skipped while the left-most column is constrained in the WHERE
  clause.


type: ci
user: drh
comment:
  Fix a faulty assert() statement.  Add comments to clarify the behavior of
  the sqlite3OpenTableAndIndices() routine in insert.c.  Add test cases to
  verify that the assert() statement is not firing inappropriately.
  Ticket [369d57fb8e5ccdff06f1].


type: t
user: drh
brief: Fixed ticket [369d57fb8e5ccdff|369d57fb8e].
comment:
  Fixed ticket [369d57fb8e5ccdff|369d57fb8e]: <i>DELETE with TRIGGER causes an assertion fault</i> plus 5 other changes


type: ci
user: drh
comment:
  Change a while-loop into a do-loop in sqlite3VdbeSerialPut() for a small
  size reduction and performance improvement.


type: ci
user: drh
comment:
  Improve the performance and reduce the size of the sqlite3VdbeSerialGet()
  routine by avoiding the use of stack.


type: ci
user: drh
comment:
  Split the sqlite3Error() routine into sqlite3Error() and
  sqlite3ErrorWithMsg(), for a slight size reduction and performance increase.


type: ci
user: drh
comment:
  Combine the pcacheAddToDirtyList() and pcacheRemoveFromDirtyList() routines
  into a single pcacheManageDirtyList() routine.  The resulting binary code is
  slightly faster and a few bytes smaller.


type: ci
user: drh
comment:
  Factor the saveAllCursors() routine of btree.c into two separate routines,
  for a noticable performance improvement.


type: ci
user: drh
comment:
  Another performance tweak: Split the sqlite3BtreeCursorHasMoved() routine 
  into two with the second routine named sqlite3BtreeCursorRestore().  The first
  now only reports whether or not the cursor has moved and the second tries to
  restore the cursor.  This allows the sqlite3VdbeCursorMoveto() routine to be
  refactored to avoid stack pointer movements, for a noticable performance gain.


type: ci
user: drh
comment:
  Make the implementation of the sqlite3_aggregate_context() interface faster
  for second an subsequent invocations.  This helps all aggregate functions to
  perform better.


type: ci
user: drh
comment:
  Changes to sqlite3ScratchMalloc() that make the entire memory allocation
  interface a little faster and about 100 bytes smaller.


type: ci
user: drh
comment:
  Fix a variable-declaration after code problem in btree.c.  Harmless in
  GCC and CLANG but unacceptable for MSVC.


type: ci
user: drh
comment:
  The sqlite3VdbeChangeEncoding() routine goes about 3x faster if the
  sqlite3VdbeMemTranslate() subroutine is not inlined.


type: ci
user: drh
comment:
  Change the name of the VdbeMemRelease() macro to VdbeMemReleaseExtern() to
  more accurately reflect what it does.  Performance enhancement to the
  sqlite3VdbeMemRelease() function.


type: ci
user: dan
comment:
  In cases where stat4 data is available but cannot be used because the rhs of a range constraint is too complex a expression, fall back to using the default estimates for number of rows scanned.


type: ci
user: drh
comment:
  Query or change the maximum number of worker threads allowed on each
  database connection separately using the "PRAGMA threads" command.


type: ci
user: drh
comment:
  Allow CAST expressions and unary "+" operators to be used in the DEFAULT
  argument of an ALTER TABLE ADD COLUMN and to be understand on the RHS of
  range constraints interpreted by STAT3/4.  This involves a rewrite of the
  implementation of the CAST operator.


type: ci
user: drh
comment:
  Test cases added for using unary "+" and CAST operators on the RHS of range
  constraints and verifying that STAT3/4 can use those constraints.


type: ci
user: drh
comment:
  Add an assert() and five testcase() macros to the OP_Cast opcode implementation
  to help verify that it is fully tested.


type: ci
user: drh
comment:
  Change the page cache so that a new sqlite3_pcache object is allocated as
  soon as the page cache is opened, not delayed until the first fetch request.
  This give a noticable performance boost.  The interface between pager and
  the page cache has changed slightly, which might break ZIPVFS.


type: ci
user: drh
comment:
  Remove the SQLITE_CONFIG_WORKER_THREADS configuration parameter.  The number
  of worker threads in the sorter is now determined only by the
  PRAGMA threads=N setting.


type: ci
user: drh
comment:
  Factor out the exception paths from sqlite3ValueToText() into a separate
  function so that the main routine is much faster for the common case of
  no required type or encoding conversions.


type: ci
user: drh
comment:
  In the sqlite3_context object, keep a pointer to the result value rather
  than storing the result value in the sqlite3_context object and using
  memcpy() to move the value back into its register
  after the function returns.  This runs faster and saves over 500 bytes
  of code space.


type: ci
user: dan
comment:
  When determining whether or not a partial index is usable, do not assume that the cursor number assigned to each table in the query is the same as its index in the FROM clause. Fix for ticket [98d973b8f5].


type: t
user: drh
brief: Fixed ticket [98d973b8f5a2ee19|98d973b8f5].
comment:
  Fixed ticket [98d973b8f5a2ee19|98d973b8f5]: <i>Partial index gives incorrect query result</i> plus 5 other changes


type: ci
user: drh
comment:
  Refactor the sqlite3PcacheFetch() routine into three separate routines,
  which are significantly faster overall and about 100 bytes smaller in
  size as well.


type: ci
user: drh
comment:
  Fix a formatting error ("%d" needed in place of "%g") on a WHERETRACE
  macro inside of the query planner.  This fix applies to debugging logic
  only.


type: ci
user: dan
comment:
  Fix a problem causing an inaccurate stat4-based estimate for the number of rows visited by a range scan.


type: ci
user: drh
comment:
  Fix a problem causing an inaccurate stat4-based estimate for the
  number of rows visited by a range scan.


type: ci
user: drh
comment:
  Disable worker threads when SQLITE_THREADSAFE=0.  Set the default compile-time
  maximum number of worker threads to 8 and honor the
  SQLITE_DEFAULT_WORKER_THREADS compile-time constant (which defaults to 0).


type: g
user: drh
comment:
  Edit [462f42af52f146fd|462f42af52]: Move to branch [/timeline?r=mistake&nd&dp=462f42af52f146fd&unhide | mistake]. Edit check-in comment.


type: ci
user: drh
comment:
  In the command-line shell, added options --lookaside, --pagecache, and
  --scratch used to configure auxiliary memories.


type: ci
user: drh
ecomment:
  Not a fault after all, rather a testing mistake.
  Move this change back to trunk.
  Was: Avoid an unnecessary OP_Move operation for expression subqueries.

comment:
  Note a fault after all, rather a testing mistake.
  Move this change back to trunk.
  Was: Avoid an unnecessary OP_Move operation for expression subqueries.


type: ci
user: drh
comment:
  Add support for using separate worker threads to speed large sorts.
  The SQLITE_MAX_WORKER_THREADS and SQLITE_DEFAULT_WORKER_THREADS compile-time
  options and the SQLITE_LIMIT_WORKER_THREADS argument to 
  sqlite3_limit() and the "PRAGMA threads=N" pragma are added.


type: ci
user: mistachkin
comment:
  For sqlite3_win32_is_nt(), assume WinRT is NT-based and revise #ifdef ordering to prefer the ANSI version of GetVersionEx, when available.


type: ci
user: drh
comment:
  Avoid a confusing (though correct) argument to the sqlite3_result_blob()
  function in the implementation of ANALYZE.


type: ci
user: drh
comment:
  Update comments in the ANALYZE command that describe how the Stat4Accum
  objecct is passed around within the VDBE.  No changes to functional code.


type: ci
user: dan
comment:
  Add a command line program that uses the extension. This serves as example code and is also useful for performance testing.


type: ci
user: dan
comment:
  Split part of "PRAGMA ota_mode" off into "PRAGMA pager_ota_mode". This allows some specialized custom VFS implementations to intercept and implement the expected pager-related effects of this pragma.


type: ci
user: dan
comment:
  Reorganize the code in sqlite3ota.c in preparation for adding support for update and delete operations.


type: ci
user: drh
comment:
  Query planner heuristic update:
  When doing a full table scan on a table that has an equality constraint on
  an unindexed column, do not allow the estimated number of output rows to
  be greater than half the total number of rows in the table.


type: ci
user: drh
comment:
  Add the sqlite3_memdebug_title_count global variable, used during debugging
  to count the number of invocations of test_memdebug_settitle.  By examining
  this variable in the debugger after a segfault, one can then set a breakpoint
  on test_memdebug_settitle that will fire just before the problem.


type: ci
user: drh
ecomment:
  Do not record a page-size change if the attempt to change the page size
  failed due to an OOM error.

comment:
  Do not require a page-size change if the attempt to change the page size
  failed due to an OOM error.


type: ci
user: drh
comment:
  Add new APIs that take 64-bit length parameters:
  sqlite3_malloc64(),
  sqlite3_realloc64(),
  sqlite3_bind_blob64(),
  sqlite3_bind_texte64(),
  sqlite3_result_blob64(),
  and sqlite3_result_texte64().
  Internal memory allocation routines also now use 64-bit unsigned length
  parameters for safety.
  Also add the sqlite3_msize() interface.
  Fix the sqlite3_get_table() to use sqlite3_realloc64() to avoid a
  integer overflow problem.


type: ci
user: drh
comment:
  Add the ".user" shell command and implement the sqlite3_user_add()
  routine.  Incremental check-in.  The code compiles but does not work.


type: ci
user: drh
comment:
  Complete the implementation of the various APIs.  Fix several problems.
  This is another incremental check-in that does not completely work.


type: ci
user: drh
comment:
  Reorder parameters on the sqlite3_user_*() interfaces for consistency.
  Add the first TCL test cases.


type: ci
user: drh
comment:
  Add support for the extra parameter on the sqlite3_set_authorizer() callback
  and support for failing an ATTACH with an authentication-required database
  using bad credentials.  The extension is now feature complete, but much
  testing and bug-fixing remains.


type: ci
user: drh
comment:
  Move user authentication blocking from sqlite3_prepare() over to the
  table lock generator, thus allowing SQL statements (like 
  "PRAGMA locking_mode") that do not touch database content to run
  prior to authentication.


type: ci
user: drh
comment:
  Fix the sqlite3_user_change() interface so that it does allow a
  non-admin user to change their own password.


type: ci
user: drh
comment:
  Enhance the sqlite3_user_add() interface to initialize the user
  authentication logic.  Add test cases for the extra argument on the
  end of the authorizer callback.


type: ci
user: drh
comment:
  Suppress the potential schema error that occurs when a non-user-auth
  SQLite library tries to parse the sqlite_user table definition in a
  user-auth database.


type: ci
user: drh
comment:
  Fix compiler warnings and change the nullMem structure initializer into a
  format that MSVC can understand.


type: ci
user: drh
comment:
  Add new APIs that take 64-bit length parameters: sqlite3_malloc64(),
  sqlite3_realloc64(), sqlite3_bind_blob64(), sqlite3_bind_text64(),
  sqlite3_result_blob64(), and sqlite3_result_text64().
  Add the sqlite3_msize() interface.
  Internal memory allocation routines now use 64-bit unsigned
  length parameters for safety.
  Fix the sqlite3_get_table() to use sqlite3_realloc64() to avoid
  a integer overflow problem.


type: ci
user: drh
comment:
  Add the SQLITE_USER_AUTHENTICATION extension to the trunk.  This extension
  is disabled by default.  Special compilation procedures are need to enable
  it.


type: ci
user: drh
comment:
  Fix an issue with sqlite3_bind_text64() with the SQLITE_UTF16 encoding
  parameter.  Remove some unreachable code from the text64() and blob64()
  implementation.


type: ci
user: drh
comment:
  Adjust comments to show that subquery flattening restriction (10) was
  removed from the code back in 2005.  This is a comment change only.


type: ci
user: dan
comment:
  Have the sqlite3_index_writer() VMs check that the final values of records inserted into indexes on rowid tables are integers.


type: ci
user: drh
comment:
  Do not flatten aggregate subqueries that contain min() or max() functions
  so that if the min()/max() are discarded by the outer query, they still
  function and cause non-aggregate expression to be evaluated on the minimal
  or maximal row.


type: ci
user: drh
comment:
  Remove the EXPENSIVE_ASSERTS in pcache.c having to do with the pSynced field
  of the Pcache object, as they are incorrect, as revealed by recent pcache
  enhancements.


type: ci
user: dan
comment:
  Have sqlite3ota.c use grave accents instead of double-quotes to enclose identifiers in generated SQL. To avoid having the SQL engine substitute a literal string if a column reference cannot be resolved.


type: ci
user: dan
comment:
  Avoid attempting to call the xFetch() method of an sqlite3_io_methods object with a version number less than 3.


type: ci
user: drh
comment:
  Reorder the elements of the Mem object for a small size reduction and
  performance improvement.

ecomment:
  Reorder the elements of the Mem object for a small size reduction and
  performance improvement.  Moved into a branch because MSVC is unable to handle
  named structure initializer on nullMem.


type: ci
user: drh
comment:
  Reduce the number of arguments to RecordCompare functions from 4 to 3,
  resulting in a small performance increase.


type: ci
user: drh
comment:
  Remove an unused parameter from sqlite3VdbeIdxRowid().  This is cosmetic
  only as the C-compiler optimizers were already omitting this parameter on
  amalgamation builds.


type: ci
user: drh
comment:
  Make sure registers are cleared properly prior to being used to store
  the result of an OP_Column operator.


type: ci
user: drh
comment:
  In the Mem object, stop requiring that Mem.xDel be NULL when the MEM_Dyn
  bit is clear.  Also reduce the amount of initialization of Mem objects.
  All for a small size reduction and performance increase.


type: g
user: drh
comment:
  Edit [0be3019ed794c10d|0be3019ed7]: Move to branch [/timeline?r=micro-optimizations&nd&dp=0be3019ed794c10d&unhide | micro-optimizations]. Edit check-in comment.


type: ci
user: drh
comment:
  Make sure that the sorting-index pre-filter recognizes that a rowid reference
  might be sortable.  This fixes a performance regression.


type: ci
user: drh
comment:
  Merge micro-optimizations into trunk after fixing the build on MSVC.
  Performance now shows 7.58% faster than the 3.8.6 release on x64 with
  gcc 4.8.1 and -Os.


type: ci
user: dan
comment:
  Modify an assert() within sqlite3PagerWalFramesize(), a function only ever used by zipvfs, to account for recent zipvfs changes.


type: ci
user: drh
comment:
  Since numeric affinity is the most common case, check it first.  Interchange
  the NONE and TEXT affinity codes for easier checking of no affinity.


type: ci
user: dan
comment:
  Add new file ext/ota/README.txt, containing notes regarding the implementation of the ota extension.


type: ci
user: drh
comment:
  Merge the Mem.r value into the MemValue union as Mem.u.r.  Hence, a Mem can
  now store an integer or a real but not both at the same time.  Strings are
  still stored in a separate element Mem.z, for now.


type: ci
user: drh
comment:
  Add the Mem.szMalloc element to the Mem object and use it to keep track of
  the size of the Mem.zMalloc allocation.


type: ci
user: drh
comment:
  Add the sqlite3VdbeMemClearAndResize() interface to be used in place of
  sqlite3VdbeMemGrow().

ecomment:
  Add the sqlite3VdbeMemClearAndResize() interface to be used in place of
  sqlite3VdbeMemGrow().  A C++ style comment was left in this check-in by
  mistake, and so it has been moved into a branch to avoid problems in any
  future bisects on windows.


type: g
user: drh
comment:
  Edit [5b9b8987797abf7c|5b9b898779]: Move to branch [/timeline?r=Cplusplus-comment&nd&dp=5b9b8987797abf7c&unhide | Cplusplus-comment]. Edit check-in comment.


type: ci
user: drh
comment:
  The OP_Column opcode runs faster and is smaller by manually in-lining the
  code that persists string values in the output register.


type: ci
user: drh
comment:
  Remove a local variable from the OP_Column implementation, resulting in a
  modest size reduction and a performance increase.


type: ci
user: drh
comment:
  Only enable HAVE_STRCHRNUL by default on linux, as that is the only place
  it appears to work by default.


type: ci
user: drh
comment:
  Do not attempt to extend the temp file if VFS version 3 is not supported and
  hence memory mapped I/O is unavailable.


type: ci
user: drh
comment:
  Fix the affinity on inserts into the ANALYZE tables.  Change the affinity
  characters to be upper case, to make the P5 parameter of comparison operators
  easier to read.


type: ci
user: drh
comment:
  Tighten the conditions under which applyNumericAffinity() be called and add
  assert() statements to prove that it is never called otherwise.


type: ci
user: drh
comment:
  Recognize the invariant that a Mem object cannot be MEM_Dyn and have 
  a non-zero szMalloc at the same time.  Enforce this with assert()s and
  exploit it in the sqlite3VdbeMemClearAndResize() routine for a performance
  increase.


type: ci
user: drh
comment:
  Simplify two conditionals and add testcase() macros to the affinity transform
  logic in the comparison operators.


type: ci
user: drh
comment:
  Enable SELECT query planning tracing when compiled with 
  SQLITE_ENABLE_SELECTTRACE and either SQLITE_DEBUG or SQLITE_TEST.


type: ci
user: drh
comment:
  Fix the usage of the _GNU_SOURCE and _BSD_SOURCE macros in the main
  internal header file, sqliteInt.h.  Set HAVE_STRCHRNUL to 1 by default on
  Linux only.


type: ci
user: drh
comment:
  Fix the SELECTTRACE_ENABLE macro so that it doesn't cause problems for
  testfixture.  Add new SELECTTRACE() calls.


type: t
user: drh
brief: Ticket [d11a6e908f8bb04d|d11a6e908f]: 7 changes
comment:
  Ticket [d11a6e908f8bb04d|d11a6e908f] <i>Query planner fault on three-way nested join with compound inner SELECT</i> status still Open with 7 other changes


type: g
user: drh
comment:
  Edit [cbe0cf9ddf46f0a6|cbe0cf9ddf]: Move to branch [/timeline?r=select-trace&nd&dp=cbe0cf9ddf46f0a6&unhide | select-trace].


type: ci
user: drh
comment:
  Experiment using linear interpolation, instead of a strict binary search,
  when looking for integer-keyed rows on a single b-tree page.  The
  experiment was not successful.  The number of key comparisons is reduced
  by about 15%, but the added complexity of the search logic causes an 
  overall reduction in performance.  The patch is saved for historical reference
  only.


type: ci
user: drh
ecomment:
  Correctly handle an ORDER BY clause on an outer query when applying
  the compound-subquery flattening optimization.  Ticket [d11a6e908f].
  Also add the SQLITE_ENABLE_SELECTTRACE option for additional debugging
  and analysis information about select statement processing.

comment:
  Correctly handle an ORDER BY clause on an outer query when applying
  the compound-subquery flattening optimization.  Ticket [89398880bcfff].
  Also add the SQLITE_ENABLE_SELECTTRACE option for additional debugging
  and analysis information about select statement processing.


type: t
user: drh
brief: Fixed ticket [d11a6e908f8bb04d|d11a6e908f].
comment:
  Fixed ticket [d11a6e908f8bb04d|d11a6e908f]: <i>Query planner fault on three-way nested join with compound inner SELECT</i> plus 3 other changes


type: ci
user: drh
comment:
  Tune the query planner to be more aggressive about using automatic indexes
  on views and subqueries for which there is not opportunity to declare a
  persistent schema index.


type: ci
user: drh
comment:
  Adjust skip-scan cost estimates slightly so that a full table scan is
  preferred over a skip-scan to a column with only two distinct values.


type: ci
user: drh
comment:
  Add the "multiplex_truncate" PRAGMA to the multiplexor extension, for
  querying and setting the truncate flag on a database connection.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Add streaming version of sqlite3changeset_apply(). Tests and fixes for the same and sqlite3changeset_start_str().


type: ci
user: drh
comment:
  Avoid calling btreeParseCellPtr() from within fillInCell() since most of
  what btreeParseCellPtr() computes is ignored by fillInCell().  Instead, have
  fillInCell() compute the values it needs inline.  Performance improvement.


type: ci
user: drh
comment:
  Add the MemPage.noPayload boolean and use it to help
  cellSizePtr() and btreeParseCellPtr() run faster.


type: ci
user: drh
comment:
  Have the clearCell() routine return the cell size to the caller, rather
  than have the caller make a separate call to cellSizePtr().


type: t
user: drh
brief: New ticket [78c0c8c3c9f7c10f|78c0c8c3c9].
comment:
  New ticket [78c0c8c3c9f7c10f|78c0c8c3c9] <i>Segfault when a DEFAULT value contains a bind parameter</i>.


type: t
user: drh
brief: Fixed ticket [78c0c8c3c9f7c10f|78c0c8c3c9].
comment:
  Fixed ticket [78c0c8c3c9f7c10f|78c0c8c3c9]: <i>Segfault when a DEFAULT value contains a bind parameter</i> plus 5 other changes


type: ci
user: drh
comment:
  Have each open database allocate its pTmpSpace when the first write cursor
  is opened, rather than on each insert or delete, for a small space savings
  and performance boost.


type: ci
user: drh
comment:
  Simplification to the random rowid picking logic that begins running when
  the maximum possible rowid has already been used.


type: ci
user: drh
comment:
  If an SQL function makes a recursive call to do an INSERT into the same
  database, make sure that the last_insert_rowid() for that INSERT is recorded.


type: ci
user: drh
comment:
  Fix the "PRAGMA integrity_check" command so that it avoids formatting error
  message context messages until it actually needs to generate an error message.
  This avoids much formatting, and hence greatly improves the performance of
  "PRAGMA integrity_check" in the common case when there are no errors.  It also
  makes the code a little smaller.


type: ci
user: drh
comment:
  Ensure that the OP_Prev opcode verifies that content has not been deleted
  out from under the cursor.  Fix for ticket [209d31e3161b9e9ff].


type: ci
user: drh
ecomment:
  Add the OPFLAG_MULTICOLUMN flag to the OP_Column opcode.  Rearrange OP_Column
  instructions to take advantage of the new flag for a small performance 
  increase

comment:
  Add the OPFLAG_MULTICOLUMN flat to the OP_Column opcode.  Rearrange OP_Column
  instrunctions to take advantage of the new flag for a small performance 
  increase


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Change the names of the stream interface APIs to be of the form
  "_strm" instead of "_str".  In other words, added an "m" to the end, to
  try to make it clear that we are talking about a "stream" and not a "string.


type: t
user: drh
brief: New ticket [209d31e3161b9e9f|209d31e316].
comment:
  New ticket [209d31e3161b9e9f|209d31e316] <i>Assertion fault when deleting a table out from under a SELECT</i>.


type: t
user: drh
brief: Fixed ticket [209d31e3161b9e9f|209d31e316].
comment:
  Fixed ticket [209d31e3161b9e9f|209d31e316]: <i>Assertion fault when deleting a table out from under a SELECT</i> plus 5 other changes


type: ci
user: drh
comment:
  Show tree diagrams of data structures in the debugging output when the 0x100
  bit is set on sqlite3WhereTrace or sqlite3SelectTrace.


type: ci
user: drh
comment:
  Remove the SQLITE_ENABLE_TREE_EXPLAIN compile-time option.  Add alternative
  debugging display routines: sqlite3TreeViewExpr(), sqlite3TreeViewExprList(),
  and sqlite3TreeViewSelect().


type: ci
user: drh
comment:
  Enable the query planner to deal with WHERE clauses that have OR terms
  nested within AND terms that are nested within OR terms.  Also remove an
  unused function declaration.


type: ci
user: drh
comment:
  Improvements to the new syntax-tree output routines:  Omit the "END SELECT"
  mark and instead terminate the graph at the last item.  Increase the maximum
  tree depth to 100.


type: ci
user: dan
comment:
  Fix a segfault that could occur if an 'optimize' command was issued on an FTS table that contained at least one row but zero terms.


type: t
user: dan
brief: New ticket [e3a290961a63239f|e3a290961a].
comment:
  New ticket [e3a290961a63239f|e3a290961a] <i>out-of-bounds write when using non-default malloc implementation and SQLITE_DIRECT_OVERFLOW_READ</i>.


type: ci
user: dan
comment:
  Avoid ever writing before the start of an allocated buffer in the DIRECT_OVERFLOW_READ code. Fix for [e3a290961a6].


type: t
user: dan
brief: Closed ticket [e3a290961a63239f|e3a290961a].
comment:
  Closed ticket [e3a290961a63239f|e3a290961a]: <i>out-of-bounds write when using non-default malloc implementation and SQLITE_DIRECT_OVERFLOW_READ</i> plus 6 other changes


type: ci
user: drh
comment:
  Avoid a NULL pointer deference when processing the IS operator if the
  right-hand side is an illegal "#ID" style variable.
  Fix for ticket [8c32a33a53092c85a15b]


type: t
user: drh
brief: New ticket [8c32a33a53092c85|8c32a33a53].
comment:
  New ticket [8c32a33a53092c85|8c32a33a53] <i>Segfault on a syntax error involving the IS operator</i>.


type: ci
user: drh
comment:
  Update to requirements marks related to changes in the memory allocation
  interface and enhancement of the documentation regarding DEFAULT clauses
  in CREATE TABLE.


type: t
user: drh
brief: Fixed ticket [8c32a33a53092c85|8c32a33a53].
comment:
  Fixed ticket [8c32a33a53092c85|8c32a33a53]: <i>Segfault on a syntax error involving the IS operator</i> plus 5 other changes


type: ci
user: drh
comment:
  Add requirements marks on the sqlite3_db_status() interface implementation.
  Fix a typo in the documentation.  Fix the new sqlite3_result_text64() routine
  so that it works correctly with an encoding parameter of SQLITE_UTF16.


type: ci
user: dan
comment:
  Improve the accuracy of the estimates used when searching an index for values not present in any stat4 samples under some circumstances.


type: ci
user: drh
comment:
  Avoid leaking Index.aiRowEst memory if an OOM causes a rollback which deletes
  the index before the aiRowEst deletion code in sqlite3AnalysisLoad() routine
  has a chance to run.  Since the aiRowEst now might be deleted from freeIndex()
  which does not always have a db pointer, make sure the aiRowEst memory is
  not held in lookaside.


type: ci
user: dan
comment:
  Improve the accuracy of the estimates used when searching an index for values not present in any stat4 samples.


type: ci
user: drh
comment:
  Enhance (and fix) the MEMTYPE tags associated with heap memory allocations
  when SQLITE_MEMDEBUG is used.


type: ci
user: drh
comment:
  Fix the corruptI.test script so that it works with
  SQLITE_ENABLE_OVERSIZE_CELL_CHECK and with SQLITE_DEFAULT_AUTOVACUUM=1.


type: ci
user: drh
comment:
  Make sure the sqlite3VdbeMemClearAndResize() routine is never called with a
  zero size parameter, since a size of zero could lead to either a memory leak
  or an assertion fault.


type: ci
user: drh
comment:
  Ensure that the Pager.pTmpSpace allocation is correct even if an OOM error
  occurs while reducing the page size.


type: ci
user: dan
comment:
  Allow FTS tokenizers to choose whether or not to consider the "*" character part of tokens or not. This restores the pre-[e21bf7a2ad] behaviour. Also fix a problem causing FTS to interpret tokens beginning with "*" characters as EOF.


type: ci
user: dan
comment:
  Change the balance_nonroot() routine to reduce the amount of memcpy work that takes place. This is a work in progress.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent trunk micro-optimizations and the DESC index GROUP BY ORDER BY
  bug fix into the sessions branch.


type: ci
user: drh
comment:
  Fix a potential problem in the whereRangeScanEst() routine when STAT4 is
  active.  The problem was introduced by recent enhancements.


type: ci
user: drh
comment:
  Add the estimated number of output rows to the EXPLAIN QUERY PLAN output
  if compiled with SQLITE_EXPLAIN_ESTIMATED_ROWS.  This feature is off by
  default for the time being.


type: ci
user: drh
comment:
  Provide the new "-config CONFIG" option to the test/releasetest.tcl script,
  which allows one to run a single test configuration selected by name.


type: ci
user: drh
comment:
  When using the -config option to releasetest.tcl, default the testing type
  to "fulltest" if it is not specified on the command line.


type: ci
user: dan
comment:
  Fix a failing test case in index5.test. Also tweak the way cache memory is divided between read-only and dirty pages when using SQLITE_CONFIG_PAGECACHE to reduce IO in some cases.


type: ci
user: dan
comment:
  Account for the ASC/DESC properties of ORDER BY expressions when using the same index for GROUP BY and ORDER BY. Candidate fix for [ba7cbfaedc].


type: t
user: drh
brief: New ticket [ba7cbfaedc7e6d96|ba7cbfaedc].
comment:
  New ticket [ba7cbfaedc7e6d96|ba7cbfaedc] <i>Wrong output order when a DESC index and GROUP BY and ORDER BY.</i>.


type: ci
user: drh
comment:
  Make sure that a GROUP BY that also implements an ORDER BY scans the table
  in the correct order.  Fix for ticket [ba7cbfaedc7e6].


type: ci
user: drh
comment:
  Require the SQLITE_ENABLE_RTREE compile-time option in speedtest1.c in order
  to enable the R-Tree tests.


type: t
user: drh
brief: Fixed ticket [ba7cbfaedc7e6d96|ba7cbfaedc].
comment:
  Fixed ticket [ba7cbfaedc7e6d96|ba7cbfaedc]: <i>Wrong output order when a DESC index and GROUP BY and ORDER BY.</i> plus 5 other changes


type: ci
user: drh
comment:
  Simplification to the insertCell() routine in btree.c, resulting in a
  performance boost and a very small size decrease.  It turns out that the
  extra work involved in sometimes avoiding an memcpy() of the first four bytes
  of a record takes more time than just unconditionally copying those
  four bytes.


type: ci
user: drh
comment:
  Remove the VdbeCursor.lastRowid cache of the current rowid, since maintaining
  the correct cache value uses more CPU cycles than just recomputing the rowid
  on the occasions when it is actually needed.  Replace it with the
  VdbeCursor.aOffset field which used to be computed from VdbeCursor.aType
  when needed. Saves 100 bytes of code space and runs 0.2% faster.


type: ci
user: drh
comment:
  Add the OP_SorterColumns opcode - an experiment in using a special case
  opcode to decode the Sorter output rather than the generic OP_Column.  This
  might be faster.  And with further work, it could eventually eliminate the
  need for OP_OpenPseudo.


type: ci
user: drh
comment:
  Reduce the CPU load imposed by sqlit3VdbeCursorMoveto() by factoring out
  some of its functions and by avoiding unnecessary calls.


type: ci
user: drh
comment:
  Remove the OPFLAG_CLEARCACHE flag from OP_Column.  In its place, change the
  P3 parameter of OP_SorterData to be the index of the pseudo-table cursor whose
  record header cache is to be cleared.  This gives a small size reduction
  and performance increase.


type: ci
user: drh
comment:
  Use the padding word in the Mem object as temporary storage for
  serial_type value in OP_Record, and thus avoid a redundant computation of
  the serial_type for each column.


type: ci
user: drh
comment:
  Make sure new sqlite3_vtab objects created by the xCreate() virtual table
  method are initialized by the system, in accordance with the documentation.


type: ci
user: dan
comment:
  Rearrange an expression in vdbemem.c to avoid a (harmless) reference to a possibly unitialized variable.


type: ci
user: drh
comment:
  Add a four-byte prefix to the BtShared.pTmpSpace buffer to avoid reading
  before the beginning of an allocation.


type: ci
user: dan
comment:
  Update the vdbe-compress.tcl script to account for expressions of the form (123>var), where "var" should be replaced with a reference to a union member.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Fix a problem causing sqlite3changeset_concat() to fail to detect attempts to concatenate patchsets which changesets.


type: ci
user: mistachkin
comment:
  Work around MSVC not being able to deduce that a local variable was initialized in a called function.


type: ci
user: drh
comment:
  If a skip-scan is a proper subset of some other scan, then adjust the
  cost of the skip-scan upward so that it is more costly than the other scan.
  Such a cost imbalance can arise under STAT4 because of difficulties in getting
  an accurate estimate for skip-scans.


type: ci
user: drh
comment:
  Further tuning of the cost estimates for skip-scan loops, especially for cases
  when skip-scan loops are in competition with regular loops.


type: ci
user: drh
comment:
  Call fsync() right after ftruncate() when in journal_mode=TRUNCATE and
  when synchronous=FULL in order to ensure that transactions are durable
  across a power loss that happens moments after the commit.  Proposed
  fix for [https://bugzilla.mozilla.org/show_bug.cgi?id=1072773].


type: ci
user: drh
comment:
  Take steps to avoid misestimating range query costs based on STAT4 data
  due to the roundoff error of converting from integers to LogEst and back
  to integers.


type: ci
user: drh
comment:
  Call fsync() right after ftruncate() when in journal_mode=TRUNCATE and
  when synchronous=FULL in order to ensure that transactions are durable
  across a power loss that happens moments after the commit.  Proposed
  fix for [https://bugzilla.mozilla.org/show_bug.cgi?id=1072773].  This
  is a cherry-pick of [3e922208b68563]


type: ci
user: dan
comment:
  Fix a problem with the parameters to an OP_Affinity in one of the VM programs generated by sqlite3_index_writer() that was causing an OOB read.


type: ci
user: dan
comment:
  Add debugging code to count the number of iterations of each loop made as part of statement execution.


type: ci
user: drh
comment:
  Change the 0x800 bit of SQLITE_TESTCTRL_OPTIMIZATIONS so that it disables
  the loading of STAT3 and STAT4 content, not just the using of that content.
  Change the internal name of that bit to SQLITE_Stat34.


type: ci
user: drh
comment:
  Implement sqlite3_create_collation() by invoking 
  sqlite3_create_collation_v2() with a NULL destructor argument.  This saves
  a little space.


type: ci
user: drh
comment:
  Add the SQLITE_ENABLE_API_ARMOR compile-time option.  This is a work in
  progress and is not yet completely functional.


type: ci
user: dan
comment:
  If a free-slot is found within a page, but using that free-slot would fragment the page further and there are already at least 60 fragmented bytes, degragment the page. This matches the behaviour of the trunk.


type: ci
user: drh
comment:
  Get the likelihood() functions working on operators like BETWEEN that
  create virtual terms in the WHERE-clause analysis.


type: ci
user: drh
ecomment:
  Enhance the automatic index logic so that it creates a partial index when
  doing so gives the same answer for less work.
  UPDATE: This change introduced a bug described by ticket [2326c258d02ead33].

comment:
  Enhance the automatic index logic so that it creates a partial index when
  doing so gives the same answer for less work.


type: ci
user: dan
comment:
  Ensure that the "Any prior cache entry associated with newKey is guaranteed not to be pinned" guarantee made to xRekey implementations is not violated.


type: ci
user: drh
comment:
  Do not use virtual (and hence redundant) WHERE-clause terms to restrict the
  content of a automatic partial index.  Show when an automatic partial index
  is used in the EXPLAIN QUERY PLAN output.


type: ci
user: drh
comment:
  Increase the resolution of the second parameter to the likelihood() SQL
  function (the probability value) so that it can handle probabilities
  as small as 0.00000001.  Formerly, it ran out of precision at 0.001.


type: ci
user: drh
comment:
  Add the SQLITE_ENABLE_API_ARMOR compile-time option for extra API parameter
  validation.  Enhance sqlite3_stricmp(), sqlite3_strnicmp(), and
  sqlite3_uri_parameter() for improved NULL parameter handling.


type: ci
user: mistachkin
comment:
  Add special handling for static mutexes in sqlite3_mutex_alloc() when automatic calls to sqlite3_initialize() are enabled.


type: ci
user: dan
comment:
  Fix a crash that could occur if the WHERE clause of an UPDATE statement on a view that does not feature a column named "rowid" contains a term such as "rowid=?".


type: ci
user: dan
comment:
  Modify the documentation for sqlite3_changes() to make it more testable. Add tests and minor fixes for the same.


type: ci
user: drh
comment:
  Update the documentation on the sqlite3_randomness() interface to conform
  to enhancements associated with the SQLITE_ENABLE_API_ARMOR change.


type: ci
user: drh
comment:
  In the OP_Column opcode, when extracting a field that is past the end of
  a short record (because the row was originally inserted prior to ALTER TABLE
  ADD COLUMN) then make sure the output register is fully NULL and does not
  contain leftover flags (such as MEM_Ephem) from its previous use.
  Fix for ticket [43107840f1c02].


type: ci
user: drh
comment:
  Fix the %c format character in sqlite3VXPrintf() so that it correctly
  handles precisions larger than 70.


type: ci
user: drh
comment:
  In the OP_Column opcode, when extracting a field that is past the end of
  a short record (because the row was originally inserted prior to ALTER TABLE
  ADD COLUMN) then make sure the output register is fully NULL and does not
  contain leftover flags (such as MEM_Ephem) from its previous use.
  Fix for ticket [43107840f1c02].  This is a cherrypick of
  check-in [24780f8ddc1683fc].


type: ci
user: drh
comment:
  Call fsync() right after ftruncate() when in journal_mode=TRUNCATE and
  when synchronous=FULL in order to ensure that transactions are durable
  across a power loss that happens moments after the commit.  Proposed
  fix for [https://bugzilla.mozilla.org/show_bug.cgi?id=1072773].
  This is a cherrypick of [3e922208b68563489].


type: ci
user: drh
comment:
  Disable the use of strchrnul() unless specifically enabled by compile-time
  options.  Cherrypick of check-in [e580470db77d6da9]


type: ci
user: drh
comment:
  Fix problems with running UPDATE and DELETE against a VIEW and referencing
  the rowid in the WHERE clause.  This is a cherrypick of [95f8ebdbf87326f2]
  and [8523670d50004f3].


type: ci
user: drh
comment: This is a cherry-pick of version [b5df5ac052].
ecomment:
  Enhance ROLLBACK so that pending queries are allowed to continue as long
  as the schema does not change.
  This is a cherry-pick of check-in [b5df5ac052].


type: ci
user: drh
comment:
  Improvements to the wording of some comments.  Reinstate an assert() that
  is only true for non-corrupt database files by adding an "|| CORRUPT_DB" term.


type: ci
user: drh
comment:
  In the balance_nonroot() routine, protect the values in aPgno[] array from
  change during the page sort, so that aPgno[] can be used to avoid unnecessary
  pointer-map updates for auto_vacuum databases.


type: ci
user: drh
comment:
  Simplify the math slightly, and reduce by one the number of loop iterations,
  for the loop in balance_nonroot() that moves cells between pages.


type: ci
user: drh
comment:
  Simplify the logic in the cell redistribution loop of balance_nonroot().
  Enhance and clarify comments and add assert() statements for additional
  verification of correctness.


type: ci
user: drh
comment:
  Change the command-line shell man-page to use the ".tr" troff directive
  instead of ".cc" for escaping the initial "." characters in the ".help"
  output.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent trunk enhancements, including the read-after-ROLLBACK change
  and the addition of sqlite3_stmt_scanstatus() support, as well as various
  minor bug fixes.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent trunk enhancements, and in particular the improvements to
  the b-tree balancing logic, into the sessions branch.


type: ci
user: dan
comment:
  If SQLITE_ENABLE_STMT_SCANSTATUS is defined, record the number of times each VDBE opcode is executed. Derive the values returned by sqlite3_stmt_scanstatus() from these records on demand.


type: ci
user: drh
comment:
  When enlarging the size of a StrAccum object, use sqlite3DbMallocSize() to
  record the entire size of the allocation, not just the requested size.


type: ci
user: drh
comment:
  Use exponential buffer size growth in StrAccum, as long as the size does not
  grow to large, to avoid excess memory allocation resize operations.  Also,
  document the fact that setting scratch memory causes SQLite to try to avoid
  large memory allocations.


type: ci
user: dan
comment:
  Add the experimental sqlite3_stmt_scanstatus() API. For comparing the number of rows actually visited by a loop with the estimate used by the query planner.


type: ci
user: drh
comment:
  Improved documentation and addition of source-code evidence marks for the
  sqlite3_config() interface.


type: ci
user: drh
comment:
  Change the definition of SQLITE_CONFIG_SCRATCH so that at most one scratch
  buffer is used per thread.  Use the generic heap memory allocator for the
  WalIterator object when running a checkpoint.


type: ci
user: drh
comment:
  Further improvements to the ".scanstats on" display in the shell.  Be sure
  to show the results of all subqueries even if there are gaps in the 
  SELECTID values.  Add ".scanstats" to the ".help" output.


type: ci
user: dan
comment:
  Add the ".scanstats on" command to the shell tool. Executing this command causes the shell tool to print values from sqlite3_stmt_scanstatus() after each query is run.


type: ci
user: drh
comment:
  Enhance whereLoopCheaperProperSubset(X,Y) so that it does not report true
  if X uses skip-scan less than Y, since in that case X might
  deserve to be cheaper even if it is a proper subset.


type: ci
user: drh
comment:
  Make sure that NULL results from OP_Column are fully and completely NULL
  and do not have the MEM_Ephem bit set.  Fix for ticket [094d39a4c95ee4].


type: t
user: drh
brief: Ticket [094d39a4c95ee4ab|094d39a4c9]: 6 changes
comment:
  Ticket [094d39a4c95ee4ab|094d39a4c9] <i>Assertion fault on CREATE TABLE ... AS SELECT ...</i> status still Open with 6 other changes


type: t
user: drh
brief: Ticket [43107840f1c02680|43107840f1]: 3 changes
comment:
  Ticket [43107840f1c02680|43107840f1] <i>Assertion fault on UPDATE</i> status still Fixed with 3 other changes


type: ci
user: drh
comment:
  Change the query planner to do a better job of estimating the number rows
  selected by a BETWEEN operator using STAT4 when both upper and lower bounds
  are contained within the same sample.


type: t
user: drh
brief: Fixed ticket [094d39a4c95ee4ab|094d39a4c9].
comment:
  Fixed ticket [094d39a4c95ee4ab|094d39a4c9]: <i>Assertion fault on CREATE TABLE ... AS SELECT ...</i> plus 3 other changes


type: ci
user: drh
comment:
  Change the SQLITE_SCANSTAT_EST parameter so that it returns a double for
  the estimated number of output rows per loop, rather than a 64-bit integer.
  Revise the output format for the ".scanstats on" in the shell to make use
  of this new capability.


type: ci
user: drh
comment:
  Changes the formatting of ".scanstats on" in the shell so that the stats for
  subqueries are grouped together and occur after the main query.


type: ci
user: drh
comment:
  On the ".scanstats on" output in the shell, initialize the estimated count for
  the first loop of each subquery to the actual loop count.


type: ci
user: drh
comment:
  Experimental changes that permit read operations to continue after a
  ROLLBACK, as long as the schema is unchanged.


type: ci
user: drh
comment:
  Added SQLITE_SCANSTAT_SELECTID.  Change the value returned by
  SQLITE_SCANSTAT_EST from sqlite3_int64 to double.  Enhanced the formatting
  and display of scan statistics using the ".scanstats on" command in the
  shell.


type: ci
user: drh
comment:
  In the ".scanstats on" output from the shell, round the estRows value to
  the nearest integer, rather than rounding toward zero.


type: ci
user: dan
comment:
  Consider using an automatic-index for a scan even if there exists a possible skip-scan that uses one or more "=" operators.


type: ci
user: dan
comment:
  When a transaction or savepoint rollback occurs, save the positions of all open read-cursors so that they can be restored following the rollback operation.


type: ci
user: drh
comment:
  Make sure that NULL results from OP_Column are fully and completely NULL and do not have the MEM_Ephem bit set. Fix for ticket [094d39a4c95ee4].


type: ci
user: drh
comment:
  Fix the %c format character in sqlite3VXPrintf() so that it correctly handles precisions larger than 70.


type: ci
user: drh
comment:
  When a transaction or savepoint rollback occurs, save the positions of all open read-cursors so that they can be restored following the rollback operation.  Cherry-pick of check-in [dd03a2802f3f27]


type: ci
user: dan
comment:
  Have calls to sqlite3_backup_init() fail if there is already a read or read-write transaction open on the destination database.


type: ci
user: drh
comment:
  Do not automatically remove the DISTINCT keyword from "a IN (SELECT DISTINCT ...)" expressions. Fix for [db87229497].


type: ci
user: dan
comment:
  Modify the documentation for sqlite3_backup_init() to indicate that it will fail if there is already a read or read-write transaction open on the destination database.


type: t
user: drh
brief: New ticket [db872294979bb80b|db87229497].
comment:
  New ticket [db872294979bb80b|db87229497] <i>Incorrect result when RHS of IN operator contains DISTINCT and LIMIT</i>.


type: ci
user: dan
comment:
  Update a couple of test cases to account for the fact that ROLLBACK does not always abort all running SELECT statements.


type: t
user: drh
brief: Fixed ticket [db872294979bb80b|db87229497].
comment:
  Fixed ticket [db872294979bb80b|db87229497]: <i>Incorrect result when RHS of IN operator contains DISTINCT and LIMIT</i> plus 5 other changes


type: ci
user: drh
comment:
  Adding the "noskipscan" token to an sqlite_stat1.stat field prevents an
  index for being used with the skip-scan algorithm.


type: ci
user: drh
ecomment:
  Unintentional edits mistakenly checked in.  Was:
  Remove an obsolete and incorrect comment from the whereLoopOutputAdjust() 
  routine in the query planner.  No changes to working code.

comment:
  Remove an obsolete and incorrect comment from the whereLoopOutputAdjust() 
  routine in the query planner.  No changes to working code.


type: ci
user: drh
comment:
  Fix a bug in the sqlite3TripAllCursors() routine that prevents it from
  reporting errors.  It is unknown at this time whether or not this omission can
  result in any incorrect result in an actual query.


type: ci
user: drh
comment:
  When a SELECT statement is terminated by a ROLLBACK TO operation, make the
  error message be "abort due to ROLLBACK" rather than 
  "callback requested query abort".


type: ci
user: dan
comment: Avoid calling sqlite3BtreeKeysize() on a b-tree cursor in SKIPNEXT or SKIPPREV state.
ecomment:
  Avoid calling sqlite3BtreeKeysize() on a b-tree cursor in SKIPNEXT or SKIPPREV state. Cherrypick of [54e7d3fcb1].


type: ci
user: drh
comment:
  Merge in all the other ROLLBACK fixes from the branch-3.8.7 branch.  
  I don't know why I was doing them one-by-one.


type: ci
user: drh
comment:
  Allow an automatic index to be used if the only uses of declared indexes for
  the same loop use the skip-scan algorithm.


type: ci
user: drh
comment:
  If a table is the right operand of a LEFT JOIN, then any column of that
  table can be NULL even if that column has a NOT NULL constraint.
  Fix for ticket [6f2222d550f5b0ee7ed].


type: ci
user: drh
comment:
  Adding the "noskipscan" token to an sqlite_stat1.stat field prevents
  an index for being used with the skip-scan algorithm.


type: ci
user: drh
comment:
  Add an ALWAYS on an always-true branch in wal.c.
  Fix the ANALYZE command so that it resets the "unordered" and "noskipscan"
  flags on indices when reloading the sqlite_stat1 table.


type: ci
user: drh
comment:
  Add some requirements marks to the record formatting logic.  Comment changes
  only - the code is unaltered.


type: ci
user: drh
comment:
  Fix the encoding of some integers to use the minimum amount of space:
  -128, -32768, -8388608, -217483648, and -140737488355328.


type: ci
user: drh
comment:
  Ensure that when the number of cells on a page drops to zero that the freelist
  and fragment counter are both cleared.  Also add evidence marks corresponding
  to file-format documentation.


type: ci
user: drh
comment:
  Remove a redundant test case (probably a copy/paste error).  Add an assert()
  to where.c to ensure that automatic indexes do not have there output row 
  counts adjusted downward by supplementary constraints.


type: ci
user: dan
comment:
  Add SQLITE_ENABLE_OTA pre-processor directives so that this branch may be compiled with or without OTA.


type: g
user: drh
comment:
  Edit [e1f893c606368134|e1f893c606]: Move to branch [/timeline?r=mistake&nd&dp=e1f893c606368134&unhide | mistake]. Mark "Closed". Edit check-in comment.


type: ci
user: drh
comment:
  Deploy heuristics (well-commented) to better estimate how much unindexed terms
  in the WHERE clause filter the number of output rows from a single table.


type: ci
user: drh
comment:
  Always reinitialized the Index.bUnordered and Index.noSkipscan flags before
  rereading the sqlite_stat1 table, even if SQLITE_ENABLE_STAT4 is defined.


type: ci
user: dan
comment:
  Fix a buffer overread during compilation of CREATE VIRTUAL TABLE statements that featured an explicit database name but no virtual table arguments. For example, "CREATE VIRTUAL TABLE main.ft USING fts4".


type: ci
user: drh
comment:
  Add the -end option to the command-line shell, which forces it to exit after
  reading prior command-line options (presumably including one or more -cmd
  options) and without reading standard input.


type: ci
user: dan
comment:
  When attempting to restart a wal file, make any required calls to sqlite3_randomness() before waiting on or checking for wal file readers. This restores the behaviour exhibited by the trunk.


type: ci
user: drh
comment:
  Remove the ill-designed "-end" option from the command-line shell.  Instead,
  allow multiple SQL or dot-commands as command-line arguments.  Any -cmd
  commands are processed first, followed by other command-line arguments, for
  backwards compatibility.


type: ci
user: dan
comment:
  Add code to parse a rank() function specification. And a tcl interface to add auxiliary functions to fts5.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Incorporate the SQLITE_CHECKPOINT_TRUNCATE enhancement and a couple of
  obscure bug fixes from trunk.


type: ci
user: drh
comment:
  Updates to the documentation for sqlite3_wal_checkpoint_v2() and related
  interfaces, including adding many requirements marks.


type: ci
user: drh
comment:
  Add code to check the validity of CollSeq objects during runtime.  This code
  was not able to detect anomalies such as came up as a result of
  ticket [e4a18565a36884b00edf66541f38c693827968ab] so it is put into a branch
  for historical reference, with the intent of leaving it out of trunk.


type: ci
user: drh
comment:
  Clarification of the meaning of the second parameter to the busy-handler
  callback.  No changes to code.


type: ci
user: drh
ecomment:
  If a table is the right operand of a LEFT JOIN, then any column of that
  table can be NULL even if that column has a NOT NULL constraint.
  Fix for ticket [6f2222d550f5b0ee7ed].

comment:
  If a table is the right operand of a LEFT JOIN, then any column of that
  table can be NULL even if that column as a NOT NULL constraint.
  Fix for ticket [6f2222d550f5b0ee7ed].


type: ci
user: drh
comment:
  Performance enhancement for single-table queries with many OR-connected
  WHERE clause terms and multiple indexes with the same left-most columns.


type: ci
user: drh
comment:
  When closing a (shared-cache) database connection, be sure to clear out
  all KeyInfo objects cached on Index objects. 
  Fix for ticket [e4a18565a36884b00edf].


type: t
user: drh
brief: Fixed ticket [6f2222d550f5b0ee|6f2222d550].
comment:
  Fixed ticket [6f2222d550f5b0ee|6f2222d550]: <i>Incorrect output on a LEFT JOIN</i> plus 6 other changes


type: ci
user: drh
comment:
  Make sure that a DISTINCT query with an ORDER BY works correctly even if
  it uses a descending index.  Fix for ticket [c5ea805691bfc4204b1cb9e].


type: t
user: drh
brief: New ticket [c5ea805691bfc420|c5ea805691].
comment:
  New ticket [c5ea805691bfc420|c5ea805691] <i>Inverted sort order when using DISTINCT and a descending index</i>.


type: t
user: drh
brief: Fixed ticket [c5ea805691bfc420|c5ea805691].
comment:
  Fixed ticket [c5ea805691bfc420|c5ea805691]: <i>Inverted sort order when using DISTINCT and a descending index</i> plus 5 other changes


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Import from trunk the fix the "test" target on the autoconf and MSVC makefiles
  and some minor compiler warning fixes.


type: ci
user: drh
comment:
  When closing a (shared-cache) database connection, be sure to clear out all
  KeyInfo objects cached on Index objects.
  Proposed fix for ticket [e4a18565a36884b00edf].


type: ci
user: drh
comment:
  The KeyInfo cache must be cleared before closing the btree, not after.
  Revised fix for ticket [e4a18565a36884b00edf].


type: t
user: drh
brief: Fixed ticket [e4a18565a36884b0|e4a18565a3].
comment:
  Fixed ticket [e4a18565a36884b0|e4a18565a3]: <i>Stale KeyInfo object used in shared-cache mode</i> plus 5 other changes


type: ci
user: drh
comment:
  Fix a (harmless) uninitialized variable reference in b-tree balancing for
  auto-vacuumed tables with overflow pages.


type: ci
user: dan
comment:
  Avoid accessing a single uninitialized byte when moving a rare 3-byte cell from an internal page to a leaf. This was not actually causing a problem, just a valgrind warning.


type: ci
user: drh
comment:
  Increase the default minimum PMA size for multi-threaded sorting from 10x
  the page size to 250x the page size.  Provide the SQLITE_SORTER_PMASZ
  compile-time option to change this default.


type: ci
user: drh
comment:
  Add the SQLITE_REVERSE_UNORDERED_SELECTS compile-time option that causes the
  "PRAGMA reverse_unordered_selects" setting to be on by default.


type: ci
user: drh
comment:
  Increase the default PMA size from 10 to 250 pages and provide the
  SQLITE_SORTER_PMASZ compile-time option to change this default.  Add
  needed mutex call when clearing the KeyInfo cache in shared-cache mode.


type: ci
user: drh
comment:
  Make the sqlite3_table_column_metadata() interface available by default and
  without requiring the SQLITE_ENABLE_COLUMN_METADATA compile-time option.
  Other sqlite3_column_* interfaces that have a run-time penalty even if they
  are unused still require the SQLITE_ENABLE_COLUMN_METADATA option at
  compile-time.


type: ci
user: drh
comment:
  Lower the default SQLITE_SORTER_PMASZ value back to 10, where it has been for
  the past couple of releases.  Applications that need a larger value can set
  one.


type: ci
user: drh
comment:
  Fix the sqlite3_table_column_metadata() routine so that it gives the correct
  answer for the "rowid" column in a WITHOUT ROWID table.  Enhance it so that
  it can be used to check for the existence of a table by setting the column
  name parameter to NULL.  The routine is now included in the build by
  default, even without the SQLITE_ENABLE_COLUMN_METADATA compile-time option.


type: ci
user: drh
comment:
  Remove the KeyInfo cache (for now - perhaps we will add it back in later - or
  maybe not since it provides negligible benefit but adds a lot of complexity
  and thread-safety risk).  Add a mutex to ATTACH to deal with a data race.


type: ci
user: drh
comment:
  Fix a bug in the threadtest4.c program.  Remove the keyinfo cache as it provides
  minimal performance improvements, and then only at SQL preparation time, not
  at runtime, and it has problems with data races in shared-cache mode.  We might
  later add the keyinfo cache back but only enable it when shared-cache mode
  is off.


type: ci
user: drh
comment:
  Make sure the Btree mutex is held when setting the locking mode and the
  secure delete flag when attaching a shared-cache database.


type: ci
user: drh
comment:
  Add new tests to the threadtest4.c program.  Fix a long-standing data race
  in WAL mode for shared-cache.


type: ci
user: drh
comment:
  Add extra tests to threadtest4.c.  Fix a benign data race accessing the
  text encoding using ENC(db).


type: ci
user: drh
comment:
  Make sure the sqlite3BtreeCount() routine does not leave index cursors in an
  inconsistent state, as doing so might result in an assertion fault inside
  of sqlite3BtreeKey() called from saveAllCursors() if content is deleted out
  from under the statement that issued the sqlite3BtreeCount() call.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge the PRAGMA data_version command and the enhancements to FK query planning
  from trunk into the sessions branch.


type: ci
user: dan
comment: Experimental opimizations to speed up FK constraint CASCADE and SET NULL action processing.
ecomment:
  Experimental opimizations to speed up FK constraint CASCADE and SET NULL action processing. Requires further testing.


type: ci
user: drh
comment:
  Simplify the implementation of the "header-value" pragmas (schema_version,
  user_version, freelist_count, and application_id) by making them more
  table-driven.


type: ci
user: drh
comment:
  Experimental "PRAGMA data_version" command for detecting when another process
  has changed the database file.


type: ci
user: drh
comment:
  Update the PRAGMA data_version command so that it reponse to changes made
  by a shared-cache database connection, and also to changes made by the same
  database connection.  Add test cases to verify the new behavior.


type: ci
user: drh
comment:
  Redefine the way PRAGMA data_version works:  It continues to change when
  any other connection commits, including shared-cache connections, but does not
  change if the local connection commits.


type: ci
user: drh
comment:
  Rework the test/releasetest.tcl script so that it uses the autoconf makefile
  instead of the test/releasetest.mk makefile.  Also add options like --dryrun
  and --buildonly.  Omit the --makefile option and replace it with --srcdir
  with the default computed relative to the releasetest.tcl script itself.


type: ci
user: mistachkin
comment:
  Permit sqlite3_shutdown() to be called with OMIT_WSD enabled and without having called sqlite3_initialize() first.


type: ci
user: drh
comment:
  Fix a potential segfault following OOM error in the test harness.  The
  SQLite core itself is not at fault.


type: ci
user: drh
comment:
  Add the threadtest target to Makefile.in.  Add --enable-load-extension to
  the configure issued by releasetest.tcl.


type: ci
user: drh
comment:
  In the releasetest.tcl script, show the test target for each configuration
  that is run.  And show the time in HH:MM:SS.


type: ci
user: drh
comment:
  Merge the new and improved releasetest.tcl script into trunk.  Add a
  "make releasetest" target to the autoconf makefile.


type: ci
user: mistachkin
comment:
  Remove 'threadtest' as a target on Windows, due to lack of pthreads.  Also, change 'fulltest' to 'fulltestonly'.


type: ci
user: dan
comment:
  When building the amalgamation with SQLITE_ENABLE_IOTRACE defined, do not mark symbol sqlite3IoTrace as static.


type: ci
user: drh
comment:
  Fix the error counter in releasetest.tcl.  And report the total time in
  HH:MM:SS instead of just seconds.


type: ci
user: drh
comment:
  Update the SQLITE_CONFIG_PAGECACHE documentation so that the maximum page
  size is correctly stated to be 65536.


type: ci
user: drh
comment:
  Fix WITHOUT ROWID tables so that they correctly deal with PRIMARY KEYs that
  contain redundant columns.


type: ci
user: drh
comment:
  Fix the "checksymbols" target in Makefile.in so that it actually works.
  Enhance the releasetest.tcl script to count the total number of tests
  run over all configurations.


type: ci
user: drh
comment:
  Round all object sizes that go into computing SQLITE_CONFIG_PCACHE_HDRSZ up
  to a multiple of 8 bytes.


type: ci
user: dan
comment:
  Change notify2.test to check that sqlite3_blocking_step() uses CPU more efficiently than sqlite3_step(), not that it results in greater overall throughput for any specific number of threads.


type: ci
user: dan
comment:
  If the sorter uses mmap'd temp files, ensure all pages of the temp file have been allocated before it is accessed. Otherwise, a disk-full condition might result in a SIGBUS exception.


type: ci
user: drh
comment:
  Update the threadtest3 test program so that its output summary is 
  compatible with releasetest.tcl.  In threadtest3, do not record errors
  that contain the string "no such table" as being fatal errors, since they
  happen sometimes in a race condition in stress1.


type: ci
user: dan
comment:
  Ensure that when a file is extended using FCNTL_SIZE_HINT the last page is allocated on disk, even if the file will only use part of it.


type: ci
user: drh
comment:
  Make sure PRAGMA data_version is updated even if the cache is empty when
  another connection changes the database.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent enhancements from trunk, including test scripts enhancements and
  the removal of limits on the number of terms in a VALUES clause.


type: ci
user: drh
comment:
  Update the command-line parsing for threadtest3 so that tests are run in the
  order they are specified on the command-line.


type: ci
user: drh
comment:
  Change threadtest3.c so that SQLITE_SCHEMA returns result in warnings
  not hard errors.  Add the Darwin-x86_64 platform to releasetest.tcl.


type: ci
user: drh
comment:
  Disable the bigsort.test module on machine with less than 8GB of available
  RAM or machine, to avoid thrashing.


type: ci
user: drh
comment:
  Fix an error in the computation of the number of hours of runtime for
  individual test runs in the releasetest.tcl script.


type: ci
user: dan
comment:
  Allow the rank column to be remapped on a per-query basis by including a term similar to "rank match 'bm25(10,2)'" in a where clause.


type: ci
user: drh
comment:
  Enhance the "lemon" executable so that it ignores -f, -W, -O, and -I
  command-line options.  This permits most of the same options that are passed
  to the compiler to also be harmlessly passed to lemon, and thus
  simplifies makefiles.


type: ci
user: drh
comment:
  Add the ability to specify an alternative compiler (clang instead of gcc)
  on the default Makefile.  Use this in releasetest.tcl to implement an
  -fsanitize=undefined test.


type: ci
user: drh
comment:
  Add logic to releasetest.tcl that detects -fsanitize=undefined errors.
  Fix a few false-positivies that come up when running the sanitize=undefined
  test.


type: ci
user: drh
comment:
  In releasetest.tcl, omit the CC=clang text from the label on Sanitize tests,
  so that the label fits on an 80-character line.


type: ci
user: drh
comment:
  Towards getting INSERT statements to except many VALUE terms (more than
  the limit imposed by SQLITE_LIMIT_COMPOUND_SELECT).  This check-in segfaults
  on a stack overflow.  And it is slow.


type: ci
user: drh
comment:
  Handle compound-select statements originating from VALUES clauses as a
  special case that does not use recursion.


type: ci
user: drh
comment:
  Enhance "INSERT INTO ... VALUES" so that the number of rows in the VALUES clause
  is not limited by SQLITE_LIMIT_COMPOUND_SELECT, and so that the stack depth
  is constant regardless of the number of rows in VALUES.


type: ci
user: dan
comment:
  Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer.


type: ci
user: dan
comment:
  Fix a problem in the unix implementation of FCNTL_SIZE_HINT on systems that do not support posix_fallocate().


type: ci
user: drh
comment:
  In releasetest.tcl, change the --dryrun option to work as it does in
  multitest.tcl of TH3.  Add the new --trace option that work like the --dryrun
  option used to work.  Add the ability to specify additional configure-script
  options in the Config array, and create configurations that use
  --disable-shared and --disable-amalgamation for testing.


type: ci
user: drh
comment:
  In releasetest.tcl, always use --disable-shared together with
  --disable-amalgamation because mingw generates a binary that crashes and
  cannot be debugged using gdb if you don't.


type: ci
user: drh
comment:
  In releasetest.tcl: add options --veryquick and pass-through compiler
  optimization settings (ex: -O0).  Collect the SQLite version number from
  the main.test output and show it during the summary report at the end.


type: ci
user: drh
comment:
  In releasetest.tcl:
  (1) Add the Failure-Detection platform.
  (2) Re-add the --disable-shared argument on Default that was mistakenly
      removed by the previous commit.
  (3) Remove the -ftrapv tests, as we read that -ftrapv does not work in GCC.


type: ci
user: drh
comment:
  Try to fix up the "valgrindtest" target in Makefile.in so that it avoids
  misuse testing that can trigger false errors.


type: ci
user: drh
comment:
  Omit modules from the "valgrind" permutation that fork off separate processes.
  Also omit selectG.test because it is timing sensitive and valgrind is too slow
  to get the right answer.


type: ci
user: drh
comment:
  Add SQLITE_ENABLE_STMT_SCANSTATUS to the Update-Delete-Limit configuration
  in the releasetest.tcl script.


type: ci
user: drh
comment:
  Change the testfixture binary so that it explicitly enabled core files 
  on a crash (on unix).  Add a test case to verify that this works.


type: ci
user: drh
comment:
  Autoconf configure script updates:  (1) remove the long-obsolete
  --enable-cross-thread-connections option.  (2) remove the --with-hints= options.
  (3) Extension loading is enabled by default.  (4) Check for strchrnull()
  (5) Update the --help text.


type: ci
user: drh
comment:
  Add the ability to put comments in the Config and Platform setup sections of
  the releasetest.tcl script.


type: ci
user: drh
comment:
  Improvements to compile-time-option hygiene.  Use "#if OPTION" instead of
  "#ifdef OPTION" in cases where that makes sense, so that -DOPTION=0 will work.
  Add the "Have-Not" configuration in releasetest.tcl which disables all of
  the "HAVE_component" compile-time options.


type: ci
user: drh
comment:
  Add detection of the isnan() library function to the configure script.  Make
  the code responsive to the HAVE_ISNAN configuration option.


type: ci
user: dan
comment:
  Optimize the unicode61 tokenizer so that it handles ascii text faster. Make it the default tokenizer. Change the name of the simple tokenizer to "ascii".


type: ci
user: mistachkin
comment:
  Harmless compiler warning fixes for 'testfixture' and the miscellaneous extensions when built with the MSVC makefile.


type: ci
user: drh
comment:
  Simplify some code in rtree, to avoid confusing the optimizer in GCC on
  some macs:
  gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00).
  Prior to these changes, compiling with -O3 would cause incorrect code to
  be generated.  The change to the nodeGetCell() routine is key.  The other
  changes are merely cosmetic details discovered while bug hunting.


type: ci
user: drh
comment:
  Makefile enhancements:  (1) Rename autoconf/tea/configure.in to
  autoconf/tea/configure.ac so that it works with the latest versions of
  autoconf.  (2) Add the "amalgamation-tarball" targets to Makefile.in and
  main.mk (renamed from "dist" in the latter case).  (3) Update the
  README.first file in autoconf/  (4) The TOP macro
  in Makefile.in is now an absolute rather than a relative path.


type: ci
user: dan
comment:
  Ensure an up to date copy of the fts5 configuration has been loaded into memory before attempting to modify the same configuration.


type: ci
user: drh
comment:
  Eliminate all use of sprintf(), strcpy() and strcat() from test logic because
  OpenBSD hates those functions.


type: ci
user: drh
comment:
  Set binary mode for output on Windows when writing a quoted string that
  might contain newline characters.


type: ci
user: drh
comment:
  Enhance the command-line shell with the ability to set the
  SQLITE_TESTCTRL_NEVER_CORRUPT flag using: ".testctrl never_corrupt 1".


type: ci
user: drh
comment:
  Add an assert() to verify that the nField+nXField values of a KeyInfo object
  are never less then the number of columns in a row for a non-corrupt database.
  This assert() currently fails, which is the root of the problem with
  ticket [f97c4637102a3ae72b].


type: t
user: drh
brief: Ticket [f97c4637102a3ae7|f97c463710]: 6 changes
comment:
  Ticket [f97c4637102a3ae7|f97c463710] <i>Incorrect ordering with ORDER BY and LIMIT</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  Improvements to the DELETE code generator for the one-pass case.  Avoid
  some OP_Goto instructions.  Read content from the index cursor if the
  index cursor is valid and was used to locate the row that is to be deleted.


type: ci
user: drh
comment:
  An alternative way of implementing the assert() that verifies the relative
  values of KeyInfo.nField+KeyInfo.nXField and the number of columns in a record.
  This version of the assert() only fires when the high-speed comparison
  routines are used - which is to say it only fires when the constraint
  actually matters.


type: ci
user: drh
comment:
  Ensure that the KeyInfo.nXField value for ephemeral tables used to implement
  ORDER BY or GROUP BY clauses is set correctly, so that the
  sqlite3VdbeFindCompare() routine can choose the correct comparison function.
  Add assert() statements to the high-speed comparison functions to detect
  cases where they are inappropriately chosen.
  Fix for ticket [f97c4637102a3ae72b7911].


type: ci
user: drh
comment:
  Make sure that the KeyInfo.nXField value of ephermeral tables used for
  ORDER BY and GROUP BY is set correctly, so that the correct comparison
  function can be choosen by sqlite3VdbeFindCompare().


type: ci
user: drh
comment:
  There are asserts in place now that will prevent a recurrence of
  ticket [f97c4637102a3ae7].  Nevertheless, it is good to add some
  test cases as well.


type: ci
user: mistachkin
comment:
  Improve precision of the new test case by removing a superfluous carriage-return from the shell command input.


type: ci
user: drh
comment:
  Fix another instance of an incorrect value for KeyInfo.nXField on a
  sorting index.  Ticket [f97c4637102a3ae72b79].


type: ci
user: drh
comment:
  Fix another instance of an incorrect value for KeyInfo.nXField on a
  sorting index.

ecomment:
  Fix another instance of an incorrect value for KeyInfo.nXField on a
  sorting index.  More changes for ticket [f97c4637102a3ae72b79].


type: ci
user: drh
comment:
  Ensure that "PRAGMA wal_checkpoint = TRUNCATE|FULL|RESTART" block on other connections and truncate the wal file as required even if the entire wal file has already been checkpointed.  Cherry-pick from trunk.


type: t
user: drh
brief: Fixed ticket [f97c4637102a3ae7|f97c463710].
comment:
  Fixed ticket [f97c4637102a3ae7|f97c463710]: <i>Incorrect ordering with ORDER BY and LIMIT</i> plus 4 other changes


type: ci
user: drh
comment:
  Change the undocumented ".selecttrace" command in the shell to accept
  an integer bitmask rather than a boolean.


type: ci
user: drh
comment:
  Make sure errors in the FROM clause of a SELECT cause analysis to abort
  and unwind the stack before those errors have a chance to mischief
  in the "*" column-name wildcard expander. Fix for ticket [32b63d542433ca67].


type: ci
user: drh
comment:
  In the command-line shell, make sure stderr is unbuffered so that it 
  automatically flushes.  This has always been the case already for unix and
  on Windows when the output is a console, but apparently was not the case
  on Windows when the output was a pipe.


type: ci
user: dan
comment:
  Have fts5 store rowids in ascending order. Query speed is virtually the same regardless of rowid order, and ascending order makes some insert optimizations easier.


type: ci
user: drh
comment:
  The va_list argument cannot take on a NULL value and cannot be compared with
  NULL on some platforms (ex: ARM).  So do not attempt to do so.


type: ci
user: dan
comment:
  Fix a problem with fts5 doclist-indexes that occured if the first rowid of the first non-term page of a doclist is zero.


type: ci
user: drh
comment:
  Fix a (almost always harmless) read past the end of a memory allocation
  that comes about because the Expr.pTab field is checked on an
  EXPR_REDUCEDSIZE Expr object before checking the Expr.op field to
  know that the Expr.pTab field is meaningless.


type: ci
user: dan
comment:
  Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function.


type: ci
user: dan
comment:
  Fix a bug in the fts3 snippet() function causing it to omit leading separator characters from snippets that begin with the first token in a column.


type: g
user: drh
comment:
  Edit [2954ab5010499684|2954ab5010]: Move to branch [/timeline?r=ota-update&nd&dp=2954ab5010499684&unhide | ota-update].


type: ci
user: dan
comment:
  When a "full", "restart" or "truncate" checkpoint is attempted, return SQLITE_BUSY if a reader prevents the checkpointer from copying any frames at all into the database file.


type: ci
user: drh
comment:
  Experimental sqlite_db_config() setting to disable writing to all btrees 
  except for one btree with a particular root page.


type: ci
user: dan
comment:
  Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query.


type: ci
user: drh
comment:
  Add an experimental pragma "PRAGMA preload;" that preloads the pcache with
  the entire database file.


type: ci
user: drh
comment:
  Add a few simple test cases for SQLITE_TESTCTRL_INITMODE - cases which
  also test PRAGMA integrity_check.


type: ci
user: dan
ecomment:
  Ensure that "PRAGMA wal_checkpoint = TRUNCATE|FULL|RESTART" block on other connections and truncate the wal file as required even if the entire wal file has already been checkpointed.

comment:
  Ensure that "PRAGMA wal_checkpoint = TRUNCATE|FULL|RESTART" block on other connections and truncate the database file as required even if the entire wal file has already been checkpointed.


type: ci
user: drh
comment:
  Split up the SRC variable in Makefile.msc to avoid over-long cmd.exe commands when TOP is set to a long pathname.


type: ci
user: dan
comment:
  Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function. Cherrypick of [245e8730451f].


type: ci
user: drh
comment:
  Change SQLITE_TESTCTRL_INITMODE to SQLITE_TESTCTRL_IMPOSTER.  Revise the order
  of parameters.  Give it the ability to reset the schema parse table so that
  imposter tables can be erased.


type: ci
user: dan
comment:
  Have ota use imposter tables to write to indexes instead of the sqlite3_index_writer() interface. The error handling in this version is broken in a few small ways.


type: ci
user: drh
comment:
  Break out the (script-generated) pragma parsing tables into a separate file,
  pragma.h, to make editing easier.


type: ci
user: drh
comment:
  Fix a typo in the --help output for speedtest1.  Fix a dependency error
  in the Makefile.in for speedtest1.


type: ci
user: dan
comment:
  Avoid a linear scan of the entire table when ota updates or deletes a row from a table with an external primary key index.


type: ci
user: drh
comment:
  Merge all changes from index_xinfo, including the move of the pragma table
  into the separate pragma.h file.


type: ci
user: drh
comment:
  Merge all recent trunk changes, including the movement of the pragma
  table into the separate pragma.h header file.


type: ci
user: drh
comment:
  Figure out the primary-key type of a table using queries of sqlite_master
  and the table_info and index_list pragmas, obviating the need for
  SQLITE_TESTCTRL_TBLTYPE.


type: ci
user: dan
comment:
  Prevent ota updates from violating NOT NULL constraints. Add a comment to the "limitations" section of sqlite3ota.h saying that CHECK constraints are not enforced.


type: ci
user: dan
comment:
  Change a comment in sqlite3ota.h to make it clear that it is not possible to insert a NULL value into an INTEGER PRIMARY KEY column using ota.


type: ci
user: drh
comment:
  Add the index_xinfo pragma which gives information about the fields that
  reference the table PRIMARY KEY in addition to the index key fields.
  Add extra columns "desc", "coll", and "key" to the index_info and index_xinfo
  pragmas.  Add the "origin" and "partial" columns to the index_list pragma.


type: ci
user: drh
comment:
  Change the name of ".info" to ".dbinfo" and add an optional second argument
  which is the ATTACH-ed DB about which information is provided.  Provide
  ".indexes" as an alternative name to the legacy ".indices" command.


type: ci
user: drh
comment:
  Fix potential 32-bit integer overflow problems on the offset and length
  parameters to sqlite3_blob_read() and sqlite3_blob_write().  For
  sqlite3_blob_open(), make sure the *ppBlob return parameter is zeroed if
  the interface fails with SQLITE_MISUSE.


type: t
user: dan
brief: New ticket [2f7170d73bf9abf8|2f7170d73b].
comment:
  New ticket [2f7170d73bf9abf8|2f7170d73b] <i>&quot;misuse of aggregate&quot; error if aggregate column from FROM subquery is used in correlated subquery</i>.


type: ci
user: drh
comment:
  In selecttrace mode 0x100, show a complete parse-tree both before and after
  query flattening.  This is a change to debugging code only.


type: ci
user: drh
comment:
  In selecttrace 0x100 mode, show the parse tree after name resolution instead
  of before flattening, so that it is always seen even if flattening does not
  occur.  Also: add the hex pointer value to the top of each SELECT tree.


type: ci
user: drh
comment:
  Disable the query flattener for aggregate subqueries if the parent query
  uses other subqueries in its result set or WHERE clause or ORDER BY clause.
  Preliminary fix for ticket [2f7170d73bf9abf8].  However it still contains
  a defect similar to the COLLATE problem of [ca0d20b6cddd].


type: ci
user: drh
comment:
  Propagate the COLLATE operator upward through function calls.
  Initial fix for ticket [ca0d20b6cdddec5e8].


type: ci
user: drh
comment:
  Propagate COLLATE operators upward through function calls and CASE operations.
  And do not flatten an aggregate subquery into a query that uses other
  subqueries.  Fixes for tickets [ca0d20b6cdddec5] and
  [2f7170d73bf9], respectively.


type: t
user: drh
brief: Fixed ticket [2f7170d73bf9abf8|2f7170d73b].
comment:
  Fixed ticket [2f7170d73bf9abf8|2f7170d73b]: <i>&quot;misuse of aggregate&quot; error if aggregate column from FROM subquery is used in correlated subquery</i> plus 5 other changes


type: t
user: drh
brief: Fixed ticket [ca0d20b6cdddec5e|ca0d20b6cd].
comment:
  Fixed ticket [ca0d20b6cdddec5e|ca0d20b6cd]: <i>COLLATE operator masked by function calls</i> plus 5 other changes


type: ci
user: dan
comment:
  Add documentation and test cases for sqlite3ota_create_vfs(). Also code to detect errors in zipvfs/ota setup.


type: ci
user: dan
comment:
  Change the way the "incremental checkpoint" function of OTA works in order to reduce the effect on the SQLite core code.


type: ci
user: drh
comment:
  Make sure the prepared statement auto-resets on extended error codes
  of SQLITE_BUSY and SQLITE_LOCKED even when compiled using
  SQLITE_OMIT_AUTORESET.


type: ci
user: mistachkin
comment:
  For the shell '.import' command, make sure the last column value present is considered before NULL filling any missing ones.


type: t
user: drh
brief: Fixed ticket [1c69be2dafc28b9f|1c69be2daf].
comment:
  Fixed ticket [1c69be2dafc28b9f|1c69be2daf]: <i>Incorrect GROUP BY when input and output columns have the same name</i> plus 3 other changes


type: ci
user: dan
comment:
  Add ota tests to increase code coverage. Fix some minor issues in error handling within the ota code.


type: ci
user: dan
comment:
  Add new file ota12.test, containing tests for applying ota updates to live databases with other active reader/writer clients.


type: ci
user: drh
comment:
  Incremental merge to prevent a huge pileup of merge conflicts resulting
  from editing the same code in two different branches.


type: ci
user: drh
comment:
  Remove a redundant call to statfs() in the xOpen() method of the unix VFS.
  Also fix an unused local variable warning.


type: ci
user: drh
comment:
  Keep track of the optimal number of reserved bytes (by looking at reserve
  byte requests in calls to sqlite3BtreeSetPageSize()) and then change the
  reserve byte count to the optimal when doing a VACUUM or when using the
  backup API.


type: ci
user: dan
comment:
  Ensure the mutex used to protect the linked list of all main database files opened by a single ota vfs is allocated.


type: ci
user: dan
comment:
  Fix some problems with resuming ota updates if saving the state is interrupted by a power failure or system crash.


type: ci
user: drh
comment:
  Update document on sqlite3_mprintf() and related functions.  Discuss the
  %w format and point out that obscure ANSI-C formats are not supported.
  No changes to code.


type: ci
user: dan
comment:
  Improve tests for resuming ota updates following power failures. Fix a problem revealed by the same.


type: ci
user: dan
comment:
  Update the sqlite3ota_db() API to account for the fact that each OTA handle now uses two SQLite database handles.


type: ci
user: drh
comment:
  This additional fix prevents a partial index from being qualified for use 
  if the constraint that qualifies the partial index is part of the ON clause of
  a LEFT JOIN.


type: ci
user: drh
ecomment:
  Make sure partial automatic indexes are not based on terms in the ON clause
  of a LEFT JOIN.  Fix for ticket [2326c258d02ead3].
  UPDATE:  This fix does not work where the partial index is a named index.

comment:
  Make sure partial automatic indexes are not based on terms in the ON clause
  of a LEFT JOIN.  Fix for ticket [2326c258d02ead3].


type: ci
user: drh
comment:
  Make sure partial indexes are not qualified incorrectly by a constraint that
  is inside the ON clause of a LEFT JOIN.  Fix for ticket [2326c258d02ead33].
  Cherry-pick from [491cfe9b3f87f].


type: g
user: drh
comment:
  Edit [c0f4e308a508183b|c0f4e308a5]: Move to branch [/timeline?r=tkt-2326c258&nd&dp=c0f4e308a508183b&unhide | tkt-2326c258].


type: ci
user: drh
comment:
  More test cases to help ensure that partial indexes do not get used if their
  qualifing constraint is inside the ON clause of a LEFT JOIN.


type: t
user: drh
brief: Ticket [2326c258d02ead33|2326c258d0]: 7 changes
comment:
  Ticket [2326c258d02ead33|2326c258d0] <i>Incorrect result when a LEFT JOIN provides the qualifying constraint for a partial index</i> status still Open with 7 other changes


type: ci
user: drh
comment:
  Make sure partial indexes are not qualified incorrectly by a constraint that
  is inside the ON clause of a LEFT JOIN.  Fix for ticket [2326c258d02ead33].


type: ci
user: dan
comment:
  Prevent partial indexes on the table on the left hand side of a LEFT JOIN from being incorrectly qualified by a constraint in the ON clause of the join. This relaxes the rule introduced by the previous commit (as the partial indexes on the table on the rhs of the LEFT JOIN may now be qualified by terms within the ON clause).


type: ci
user: drh
comment:
  Remove an always-false conditional from constructAutomaticIndex().  Put an
  assert() in its place to prove that the conditional is always false.


type: t
user: drh
brief: Fixed ticket [2326c258d02ead33|2326c258d0].
comment:
  Fixed ticket [2326c258d02ead33|2326c258d0]: <i>Incorrect result when a LEFT JOIN provides the qualifying constraint for a partial index</i> plus 3 other changes


type: ci
user: drh
comment:
  Make sure the sqlite3_mutex.id field is initialized in the Win32
  mutex implementation, even when SQLITE_DEBUG is turned off.


type: ci
user: drh
comment:
  Fix a real bug (in test code) that was introduced while trying to eliminate
  harmless compiler warnings from OpenBSD (see check-in [10321910990195878c]).


type: ci
user: dan
comment:
  Fix suffix and prefix compression of terms in top-level fts5 segments. And a crash that could follow an OOM condition.


type: ci
user: drh
comment:
  In the command-line shell, the inability to read ~/.sqliterc is no longer
  a fatal error.  A warning is issued, but processing continues.


type: ci
user: drh
comment:
  Remove all references to SQLITE_DEFAULT_TEMP_CACHE_SIZE.  Add requirements
  marks related to cache_size changing.


type: ci
user: drh
comment:
  Ensure that automatic indexes are *not* considered when doing the the subqueries
  of the OR-optimization.


type: ci
user: drh
comment:
  Changes so that the amalgamation and the command-line shell build without
  modifications on VxWorks 7.  Still gives a few compiler warnings, and still
  mostly untested on that platform.


type: ci
user: drh
comment:
  Enhance SQLITE_ALLOW_URI_AUTHORITITY to elide //localhost/ and to deal
  with dodgy URIs that contain too many "/" characters.


type: ci
user: drh
comment:
  Clean up #ifdef logic dealing with VxWorks in os_unix.c.  On VxWorks, 
  automatically use posix advisory locking if it is available or fall back
  to named semaphore locking if not.


type: ci
user: dan
comment:
  Fix the bm25() function so that it multiplies scores by -1 before returning them. This means better matches have a lower numerical score, so "ORDER BY rank" (not "ORDER BY rank DESC") does what you want.


type: ci
user: drh
comment:
  Revert "PRAGMA index_info" to output only three columns, for complete
  compatibility with prior versions.  The new "PRAGMA index_xinfo" can be
  used to get the extra information in 4th, 5th, and 6th columns.


type: ci
user: dan
comment:
  Add comments to sqlite3ota.h to make it clear that passing NULL in place of a parent VFS name to sqlite3ota_create_vfs() causes the new VFS to use the system default as its parent.


type: ci
user: dan
comment:
  Fix some problems with OTA and empty target databases, or target databases with the wrong set of tables. Also add SQLITE_FCNTL_VFSNAME support to the OTA VFS.


type: ci
user: drh
comment:
  The LIKE optimization must be applied twice, once for strings and a second
  time for BLOBs.  Ticket [05f43be8fdda9f].  This check-in is a proof-of-concept
  of how that might be done.


type: t
user: drh
brief: New ticket [05f43be8fdda9fbd|05f43be8fd].
comment:
  New ticket [05f43be8fdda9fbd|05f43be8fd] <i>Incorrect use of index with LIKE operators when the LHS is a blob</i>.


type: ci
user: drh
comment:
  Refactor some jump opcodes in the VDBE.  Add JumpZeroIncr and DecrJumpZero.
  Fix the LIKE optimization to work with DESC sort order.


type: ci
user: drh
comment:
  Fix the LIKE optimization so that it finds BLOB entries in addition to text
  entries.  Ticket [05f43be8fdda9f].


type: t
user: drh
brief: Fixed ticket [05f43be8fdda9fbd|05f43be8fd].
comment:
  Fixed ticket [05f43be8fdda9fbd|05f43be8fd]: <i>Incorrect use of index with LIKE operators when the LHS is a blob</i> plus 5 other changes


type: ci
user: drh
comment:
  Fix the LIKE optimization so that it works even if there are additional
  range contraints on the column that is subject to the LIKE or GLOB.


type: ci
user: drh
comment:
  Merge all recent trunk changes, and in particular the blocking WAL lock change,
  into the apple-osx branch.


type: ci
user: drh
comment:
  Revise test cases in malloc5.test to accommodate varying allocation
  sizes returned by some system malloc() implementations.


type: ci
user: drh
comment:
  Arrange for some of the transient locks in WAL mode to block, as a single
  to the OS to fix priority inversions.


type: ci
user: dan
comment:
  Allow the query planner to evaluate deterministic scalar SQL functions used in WHERE constraints if all arguments are SQL literals in order to compare the results with sqlite_stat4 sample data.


type: ci
user: drh
comment:
  Expand the multi-process test cases to repeat each case 20 times and
  to repeat tests using different journal modes.


type: ci
user: dan
comment:
  When scanning the full-text index as part of the fts5 integrity-check, also run a point query for every term and verify that these results are consistent with those found by the linear scan.


type: ci
user: dan
comment:
  Allow the default value for columns added using ALTER TABLE ADD COLUMN to be a function in existing schemas loaded from disk. But prevent this version of SQLite from being used to create such a column.


type: ci
user: drh
comment:
  If an error occurs in the compile-time evaluation of an application-defined
  function, then propagate back out the exact error code, not just the
  generic SQLITE_ERROR.


type: ci
user: drh
comment:
  When STAT4 is enabled, allow probes of the STAT4 table using the value
  of constant functions computed at compile-time.


type: ci
user: dan
comment:
  Add tests to ensure "PRAGMA incremental_vacuum" and "PRAGMA auto_vacuum = incremental" handle corrupt databases correctly.


type: ci
user: dan
comment:
  When estimating the number of rows visited by a range scan for which the keys consist of more than one field, consider prefixes of stat4 samples as well as the full samples.


type: ci
user: dan
comment:
  When estimating the number of rows visited by a range scan for which the keys consist of more than one field, consider prefixes of stat4 samples as well as the full samples. This generates more accurate estimates.


type: ci
user: drh
comment:
  When a WHERE clause contains disjuncts with the same operands, try to
  combine them into a single operator.  Example:  (x=A OR x&gt;A) becomes
  (x&gt;=A).


type: ci
user: drh
comment:
  When the WHERE clause contains two OR-connected terms with identical
  operands but different operators, try to combine them into a single term.
  Example:  (X=A OR X&gt;A) becomes (X&gt;=A).


type: ci
user: drh
comment:
  When deleting the master journal to commit a multi-database transaction, do
  not sync the directory if PRAGMA synchronous=OFF for all participating
  database files.


type: ci
user: drh
comment:
  Modify the fts3query.test script so that it works even when testfixture is
  built using a version of TCL that is unable to sort the
  integer -9223372036854775808


type: ci
user: drh
comment:
  Merge all recent trunk enhancements and especially the blocking WAL-lock fix
  into the apple-osx branch.


type: ci
user: dan
comment:
  Allow whereShortCut() to use the PRIMARY KEY index of a WITHOUT ROWID table to optimize a vector of "IS" operators in a WHERE clause.


type: ci
user: dan
comment:
  Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer.


type: ci
user: dan
comment:
  Fix an FTS3/4 problem with handling empty tokenizer declarations (e.g. "CREATE VIRTUAL TABLE t(tokenize=);").


type: ci
user: dan
comment:
  Fix a problem with creating virtual table with names specified using malformed utf-8 within utf-16 databases.


type: ci
user: dan
comment:
  Fix a crash that can occur following an OOM condition within a CREATE VIRTUAL TABLE statement on a utf-16 database.


type: ci
user: drh
comment:
  If a virtual table is created with a malformed UTF8 name in a UTF16 database,
  make sure that does not cause problems.


type: g
user: drh
comment:
  Edit [9969cff2d0553c9b|9969cff2d0]: Move to branch [/timeline?r=malformed-vtab-name&nd&dp=9969cff2d0553c9b&unhide | malformed-vtab-name].


type: ci
user: dan
comment:
  Fix a problem causing collation sequence names to be dequoted multiple times under some circumstances.


type: g
user: drh
comment:
  Edit [e750830f1e61160c|e750830f1e]: Move to branch [/timeline?r=seek-eq&nd&dp=e750830f1e61160c&unhide | seek-eq].


type: ci
user: dan
comment:
  Disable the sorter optimization used by INSERT INTO SELECT statements if the statement explicitly specifies REPLACE, IGNORE or FAIL conflict handling.


type: ci
user: dan
comment:
  Where possible insert the set of new keys for each index in sorted order within "INSERT INTO ... SELECT" statements.


type: ci
user: mistachkin
comment:
  Enhance the shell export to support emitting column names in 'insert' mode when headers are enabled.


type: ci
user: drh
comment:
  Correctly detect the error of having a "*" wildcard on a SELECT without
  a FROM clause on the left-hand side of a recursive CTE.


type: ci
user: drh
comment:
  If a column is both UNIQUE and a PRIMARY KEY, make sure the PRIMARY KEY
  designation takes precedence.


type: ci
user: dan
comment:
  Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test.


type: ci
user: dan
comment:
  Improve fts5 integrity-check so that it checks that DESC queries return the same as ASC. Change the poslist format slightly to make room for a delete-flag.


type: ci
user: dan
comment:
  Change walblock.test to block an external process for 10 seconds, not 5. 5 seconds is not long enough to tell the difference between a blocking lock and a series of retries.


type: ci
user: drh
comment:
  Do not invoke a C preprocessor macro with an empty argument, as (reportedly)
  some versions of GCC are unable to deal with that.


type: ci
user: drh
comment:
  Add the sqlite3_status64() interface.  Make the new interface and the legacy
  sqlite3_status() both atomic and threadsafe.  Check threadsafety using
  assert()s.


type: ci
user: dan
comment:
  When the OTA module updates or deletes a row, save only those fields that are part of an index or primary key to the ota update database.


type: ci
user: drh
comment:
  Track total memory usage using a 64-bit integer on 64-bit systems.  Add
  the sqlite3_status64() interface.  Make the sqlite3_status() and
  sqlite3_status64() interfaces atomic using mutexes and verify correct
  mutex operation using assert() statements.


type: ci
user: drh
comment:
  Simplify the EXPLAIN output of virtual table P4 parameters to only show the
  pointer to the sqlite3_vtab object and omit the sqlite3_module object.


type: ci
user: drh
comment:
  Prevent a virtual table from being destroyed while it is in use.
  Also: replace Vdbe.inVtabMethod with sqlite3.nVDestroy.  Simplify the
  EXPLAIN output for P4.pVtab to only show the sqlite3_vtab pointer.


type: ci
user: drh
comment:
  Merge the latest trunk enhancements, and especially the fix for the infinite
  loop on a DROP TABLE in a corrupt database, into the apple-osx branch.


type: ci
user: dan
comment:
  Optimize CREATE INDEX, REINDEX and VACUUM statements by taking better advantage of the fact that index keys are being inserted into b-trees in sorted order.


type: g
user: drh
comment:
  Edit [cbb93d1aa1f992f1|cbb93d1aa1]: Move to branch [/timeline?r=stdcall-3.8.8&nd&dp=cbb93d1aa1f992f1&unhide | stdcall-3.8.8].


type: ci
user: drh
comment:
  Fix the saveCursorPosition() routine in btree.c so that it works
  correctly for a eState=CURSOR_SKIPNEXT cursor.


type: ci
user: dan
comment:
  Optimize cases where all the sorter is sorting a set of records that all begin with integer values, or that all begin with text values to be compared using BINARY.


type: ci
user: drh
comment:
  Provide source code line numbers for errors in the Windows VFS generated
  by the winLogIoerr() routine.


type: ci
user: drh
comment:
  Avoid leaving the malloc subsystem in a partially initialized state if
  the low-level initialization callback fails.


type: ci
user: mistachkin
comment:
  Using MSVC, get the sqlite3IoTrace variable working with the USE_STDCALL and/or DYNAMIC_SHELL options.


type: ci
user: drh
comment:
  Fix the ".testctrl imposter" command in the command-line shell to be
  consistent in error reporting and results with the other .testctrl
  commands.


type: ci
user: drh
comment:
  Detect and suppress an endless loops in clearDatabasePage() that might result
  from a corrupt database file.  This is an edited cherry-pick from
  [30011ad2f55c] and [395bb3e677a].


type: ci
user: drh
comment:
  Ensure that comparison operators do not mess up the MEM_Dyn flag on registers
  when reverting affinity changes.


type: ci
user: drh
comment:
  Change the sqlite3_log() message generated on a retry after delay in os_win
  to be an SQLITE_NOTICE instead of SQLITE_IOERR.


type: ci
user: drh
comment:
  Add the "mptest" target to main.mk.  Change TRUNCATE and PERSIST to DELETE
  for mptest.c on Windows because of problems trying to delete the journal.


type: g
user: drh
comment:
  Edit [c8694657bdc5d091|c8694657bd]: Move to branch [/timeline?r=winTest&nd&dp=c8694657bdc5d091&unhide | winTest].


type: ci
user: drh
comment: Improved tracing and debugging support on windows.
ecomment:
  The build is still broken.  Multiple definitions of sqlite3OSTrace when compiled
  using separate source files.


type: g
user: drh
comment:
  Edit [5c965a71b957fdb0|5c965a71b9]: Move to branch [/timeline?r=winTest&nd&dp=5c965a71b957fdb0&unhide | winTest]. Edit check-in comment.


type: g
user: dan
comment:
  Edit [763d2bc74b560c86|763d2bc74b]: Move to branch [/timeline?r=mistake&nd&dp=763d2bc74b560c86&unhide | mistake]. Add propagating "hidden".


type: ci
user: dan
comment:
  When vacuuming an index that uses no collations other than BINARY, assume that the order of index entries will not be changed by the VACUUM.


type: ci
user: dan
comment:
  Reduce the CPU used by CREATE INDEX statements by taking better advantage of the fact that keys are inserted in sorted order.


type: ci
user: drh
comment:
  Disable the SQLITE_FCNTL_WAL_BLOCK feature for now.  It needs more work and
  is not yet ready for release.


type: ci
user: dan
comment:
  Fix a problem with resolving ORDER BY clauses that feature COLLATE clauses attached to compound SELECT statements.


type: ci
user: drh
comment:
  Add the --msvc command-line option to the releasetest.tcl script.  This is
  only a partial merge from the winTest branch as the OS_TRACE changes are
  omitted.


type: ci
user: dan
comment:
  Fix a problem with fts3 prefix terms within phrase queries on "order=DESC" tables with a mix of negative and positive rowids.


type: ci
user: drh
comment:
  Avoid signed integer overflow when converting oversized in-line integer
  widths and precisions in printf().


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Fix problems in the sessions module causing it to produce spurious SQLITE_NOMEM errors when handling SQL text or blob values zero bytes in size.


type: ci
user: drh
comment:
  Remove two pointless assert() statements.  This should silence harmless
  compiler warnings reported at 
  [https://bugzilla.mozilla.org/show_bug.cgi?id=1152845]


type: ci
user: drh
comment:
  Add the --primarykey option to the sqldiff tool, which causes it to use the
  schema-defined PRIMARY KEY.


type: ci
user: drh
comment:
  Add the BtCursor.pPage field which is the current page to which the cursor
  points, for a very small performance gain.


type: ci
user: dan
comment:
  In sqlite3_declare_vtab(), avoid accessing the database structure until after the "api-armour" safety-check has completed and the db mutex has been obtained.


type: ci
user: drh
comment:
  Fix foreign key CASCADE for cases where the parent key is an INTEGER PRIMARY
  KEY and the parent table contains other columns named "rowid", "_rowid_",
  and "oid".


type: ci
user: drh
comment:
  Work toward adding the --changeset option to the sqldiff utility program.
  Changes are incomplete.  This is an incremental check-in.


type: ci
user: dan
comment:
  Do not assume an index contains unique entries unless it is declared UNIQUE and NOT NULL is specified for all columns. Fix for [7b4fee9f6c].


type: t
user: dan
brief: New ticket [7b4fee9f6c20b634|7b4fee9f6c].
comment:
  New ticket [7b4fee9f6c20b634|7b4fee9f6c] <i>Expressions like (a IS NULL AND b = ?) optimized by a UNIQUE index matching a single row only</i>.


type: t
user: dan
brief: Closed ticket [7b4fee9f6c20b634|7b4fee9f6c].
comment:
  Closed ticket [7b4fee9f6c20b634|7b4fee9f6c]: <i>Expressions like (a IS NULL AND b = ?) optimized by a UNIQUE index matching a single row only</i> plus 6 other changes


type: ci
user: drh
comment:
  Add the --changeset option to the sqldiff utility program, for generating
  a sessions changeset file instead of an SQL diff.


type: ci
user: dan
comment:
  Have fts5 integrity check verify that prefix indexes contain the same values as returned by prefix queries on the main terms index.


type: ci
user: drh
comment:
  Update the version number to 3.8.10 because of the new sort optimizations from check-in [79326d6eec].


type: ci
user: drh
comment:
  Fix a problem with a stale Select.pWith pointer when a CTE is used together
  with a compound query and an ORDER BY clause.


type: ci
user: dan
comment:
  Test that if the definition of a collation sequence is changed and VACUUM run, the new database contains records sorted in the (new) correct order.


type: ci
user: drh
comment:
  Remove the out2-prerelease VDBE opcode property and its associated code,
  for a 0.5% performance improvement.


type: ci
user: drh
comment:
  On a DETACH statement, keep all schemas intact except fo the one that is
  being detached, and thus avoid unnecessary schema reparsing.


type: ci
user: drh
comment:
  Add #ifdef magic for HAVE_GETHOSTUUID so that the build will hopefully now
  work on more verions of MacOS with SQLITE_ENABLE_LOCKING_STYLE turned on.


type: ci
user: drh
comment:
  An oversize hex literal can cause a parsing error while generating code for
  constants that are factored out of the main body of the VDBE program.  So
  allow for that case.


type: ci
user: drh
comment:
  Add a test case for what was formerly thought to be an unreachable condition:
  when the LHS of an OR operator contains an error and the RHS contains an IN
  operator.


type: ci
user: drh
comment:
  When doing the (dubious) conversion of double-quoted identifier into string
  literal for MySQL compatibility, be sure to also clear the iTable value from
  the Expr entry to avoid an assert.


type: ci
user: drh
comment:
  When adding the implied "LIMIT 1" to the end of a scalar subquery, make sure
  that subquery is not a VALUES-only query as such queries cannot deal with
  LIMIT clauses.


type: ci
user: drh
comment:
  Fix a faulty assert() statement in the name resolver associated with the
  optimization that converts compound selects with ORDER BY COLLATE into
  subqueries.


type: ci
user: drh
comment:
  Fix the error message generator for illegal token errors so that it does not
  leak memory if it immediately follows another erroneous SQL statement.


type: ci
user: drh
comment:
  Enhance the showdb utility program so that it can read the last partial page
  of a truncated database file.


type: ci
user: drh
comment:
  Prevent the fetchPayload() routine from reporting a cell size that extends
  off the end of the page on a pathologically corrupted database file.


type: ci
user: dan
comment:
  Logically store updates as (insert+delete) within the FTS tree. This allows keys to be annihilated more quickly under some circumstances.


type: ci
user: drh
comment:
  Use a heap rather than a bitmap for cell coverage and overlap testing on
  btree pages in PRAGMA integrity_check.


type: ci
user: drh
comment:
  Use a heap instead of a bitmap for cell overlap and coverage testing of
  btree pages in PRAGMA integrity_check.


type: g
user: drh
comment:
  Edit [4b0f44848dfcf225|4b0f44848d]: Move to branch [/timeline?r=mistake&nd&dp=4b0f44848dfcf225&unhide | mistake]. Edit check-in comment.


type: ci
user: drh
comment:
  Silently ignore requests to change the PRAGMA synchronous setting to an
  illegal integer value, rather than raising an assertion.


type: ci
user: drh
comment:
  Fix the VDBE so that it always uses the original opcode for profiling and
  debugging even after the pOp pointer has been updated due to a jump.


type: ci
user: drh
comment:
  Make sure all cursors are closed when returning from a VDBE subprogram that
  implements a foreign-key construct.


type: ci
user: dan
comment:
  Have OTA always specify SQLITE_CONFIG_URI when opening databases. Fix a test issue causing otacrash.test to fail.


type: ci
user: drh
comment:
  Ensure that semantic SQL errors are always reported back up to the syntax
  parser.  Also:  Improve the defense against invalid PRAGMA synchronous settings.


type: ci
user: drh
comment:
  Remove unnecessary parser error count increments.  Let the sqlite3ErrorMsg()
  take care of doing that.


type: ci
user: drh
comment:
  In the expression-tree comparison routine, do not compiler Expr.iColumn
  and Expr.iTable for TK_STRING expressions.


type: ci
user: dan
comment:
  Ensure that if a "ROLLBACK TO" statement is used to rollback (but not close) theoutermost transaction, the xRollbackTo() method of any virtual tables involved in the transaction is invoked.


type: ci
user: drh
comment:
  The sqlite3_trace() callback does not try to expand parameters in statements
  that have no parameter.


type: ci
user: drh
comment:
  Bring comments on the INSERT code generator up-to-date.  Fix the INSERT code
  generator so that it correctly handles inserts from a SELECT into a virtual
  table with non-terminal hidden columns.


type: ci
user: drh
comment:
  Silently ignore parser stack overflow when parsing a malformed schema in
  PRAGMA writable_schema mode.


type: ci
user: drh
comment:
  Fix a broken assert() and comparison for INSERT INTO ... SELECT when the
  SELECT uses an unknown collating sequence.


type: ci
user: drh
comment:
  Enhance fuzzershell to support multiple blocks of SQL, each run in its own
  private in-memory database.


type: ci
user: dan
comment:
  Fix an obscure problem with "INSERT INTO tbl(cols) SELECT" statements where the SELECT is a compound with an ORDER BY and "cols" is a strict subset of tbl's columns.


type: ci
user: drh
comment:
  Fix a memory leak in the SELECT code generator tracing logic (code that is
  omitted in non-debugging builds).


type: ci
user: drh
comment:
  Make sure the datatype and collating sequence of transients tables are
  not initialized more than once.  Otherwise a memory leak results.


type: ci
user: drh
comment:
  Alternative fix for the double-initialization of type and collating sequence
  problem that leaves the estimated table size intact.


type: ci
user: drh
comment:
  Change the printf, strftime, and glob modes of fuzzershell so that they
  reject all inputs that do not start with a string literal followed by a
  comma.  This helps the fuzzer focus in on the kinds of behavior those modes
  are intended to test.


type: ci
user: dan
comment:
  Have OTA maintain a small LRU cache of UPDATE statements. This reduces the amount of time it spends compiling UPDATE if a single data_xxx table contains many different ota_control strings.


type: ci
user: dan
comment:
  Fix a problem in the "ota" utility program causing it to crash if SQLite failed to open the target database.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Modify the sqlite3session_diff() API so that tables with no PRIMARY KEYs are ignored. This matches the other sessions APIs. Also change sqlite3session_diff() so that it returns SQLITE_SCHEMA, instead of SQLITE_ERROR, if the tables being compared do not have compatible schemas.


type: ci
user: dan
comment:
  Fix a performance problem in calls to sqlite3ota_close() made after the OTA update has been completely applied and checkpointed.


type: ci
user: drh
ecomment:
  Oops - the deephemerialization needs to occur before the move, not after.  Was:
  Rather than try to keep track of the parent of an ephermeral register when
  the register moves, just deephemeralize all registers touched by the OP_Move
  opcode.

comment:
  Rather than try to keep track of the parent of an ephermeral register when
  the register moves, just deephemeralize all registers touched by the OP_Move
  opcode.


type: ci
user: drh
comment:
  Fuzzershell enhancements:  (1) Add the --verbose and --quiet flags
  (2) Show percentage complete and final test count for multi-test inputs
  (3) Omit trace and result logs unless the --verbose flag is used.


type: ci
user: drh
comment:
  Add AFL-generated test cases in the test/fuzzdata1.txt file.  Automatically
  run fuzzershell against those cases on a "make test".


type: ci
user: drh
comment:
  In the fuzzershell, always invoke the trace and log callbacks even if output
  is suppressed.  Keep track of the current test name in a global variable
  for simplified debugging.


type: ci
user: drh
comment:
  In fuzzershell: (1) comment fixes.  (2) Set and clear g.zTestName[] correctly.
  (3) Use the value in g.zTestName[] in error messages.


type: ci
user: drh
comment:
  Fuzzershell: change the error summary output to work with releasetest.tcl.
  Reduce the maximum number of OOM interations.


type: ci
user: dan
comment:
  Change the fts5 content= option so that it matches fts5 columns with the underlying table columns by name, not by their position within the CREATE TABLE statement.


type: ci
user: drh
comment:
  Shift the Mem.pScopyFrom pointer in the correct direction in OP_Move.
  Fix for a defective check-in [a2135ad13049] and ticket [d6352363751].
  Debugging code only - does not affect normal operation.


type: ci
user: drh
comment:
  Add the SQLITE_LIMIT_PRINTF_WIDTH setting for sqlite3_limit() and associated
  logic for preventing DOS attacks using printf() with oversized widths or
  precisions.


type: g
user: drh
comment:
  Edit [091cfe8e0a89c64f|091cfe8e0a]: Move to branch [/timeline?r=mistake&nd&dp=091cfe8e0a89c64f&unhide | mistake]. Mark "Closed". Edit check-in comment.


type: ci
user: dan
comment:
  Fix an assert() failure that could occur if the internal sqlite3_rename_parent() SQL scalar function was invoked directly.


type: ci
user: drh
comment:
  Fix sqlite3VdbeMakeWritable() to always clear the MEM_Ephem flag.
  Change the OP_Move opcode to always deephemeralize moved content.


type: ci
user: drh
comment:
  Fix signed/unsigned comparison compiler warnings.  Add the
  SQLITE_OMIT_RANDOMNESS compile-time option to cause the PRNG to be seeded
  identically on every run, for testing purposes.


type: ci
user: drh
comment:
  Modify the mkfuzzdata1.tcl script so that it retains the culled test vectors
  in the tmp2 subdirectory.


type: ci
user: drh
comment:
  Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3,
  and to ensure that error messages are not overwritten and thus leaked.


type: ci
user: dan
comment:
  Fix an assert() failure caused by setting "PRAGMA journal_mode=off" on an database in exclusive-mode that has already opened the journal file.


type: ci
user: drh
comment:
  Enhance the fuzzershell --uniquecases option to output results in order of
  increasing runtime and to include the runtime in the comment separator of
  the output.


type: ci
user: dan
comment:
  Reorganize some multi-threaded code in vdbesort.c so that full MC/DC test coverage does not depend on the outcome of a race condition.


type: ci
user: drh
comment:
  Improvements to fuzzershell: Avoid excess memory allocations when loading
  many files.  Show the total runtime on final output.  Show individual filenames
  as they are processed even if they are single test-case files.


type: ci
user: dan
comment:
  Change to storing all keys in a single merge-tree structure instead of one main structure and a separate one for each prefix index. This is a file-format change. Also introduce a mechanism for managing file-format changes.


type: ci
user: drh
comment:
  Fix a bug in --changeset output for sqldiff.exe for floating-point values.
  Also fix harmless compiler warnings in sqldiff.exe.


type: ci
user: drh
comment:
  Rename the test_stat.c source file to dbstat.c and rework the makefiles to
  make dbstat.c a first-class source module.


type: ci
user: drh
comment:
  Add the "dbstat" virtual table as a standard component in the SQLite
  amalgamation, though turned off unless compiled with SQLITE_ENABLE_DBSTAT_VTAB.


type: ci
user: drh
comment:
  Automatically enable the dbstat virtual table on all new database
  connections with the SQLITE_ENABLE_DBSTAT_VTAB compile-time option is used.


type: ci
user: drh
comment:
  Fix the sqlite3_memory_used() and sqlite3_memory_highwater() interfaces so
  that they really do provide a 64-bit answer.


type: ci
user: dan
comment:
  Change autoconf/Makefile.am to avoid building target sqlite3.o as part of both the shared library and shell tool. Doing so causes problems for parallel builds.


type: ci
user: drh
comment:
  Build the sqlite3_analyzer.exe and sqldiff.exe programs on "make test"
  and "make smoketest" and other similar test targets.


type: ci
user: drh
comment:
  Test cases for sqlite3_analyzer and sqldiff.  Fix a problem with 
  sqlite3_analyzer related to the renaming of the initialization routine.


type: ci
user: drh
comment:
  Add missing "finish_test" commands to the end of the two new test scripts
  for sqlite3_analyzer and sqldiff.


type: ci
user: drh
comment:
  Try to get recent sqlite3_analyzer and sqldiff tests working for all
  tested combinations of compile-time options, especially
  SQLITE_OMIT_VIRTUALTABLE and SQLITE_OMIT_LOAD_EXTENSION.


type: ci
user: drh
comment:
  Experimental API extension for recovering the current trace and profile
  callbacks on a database connection.


type: ci
user: drh
comment:
  An early attempt to get indexes to work with the IS operator.  This code
  passes tests, but much more testing is needed to verify that it works on
  all corner cases.


type: ci
user: dan
comment:
  Change fts5 doclist-index structures to be trees instead of flat lists. This only makes a difference for databases that contain millions of instances of the same token.


type: ci
user: drh
comment:
  A new implementation of indexing with the IS operator that works correctly
  when the IS operator is in the WHERE clause and the operands are from 
  opposite sides of a LEFT JOIN.


type: ci
user: dan
comment:
  Merge changes from the index-is-operator branch into this one. Drop the partial support added for IS on this branch by [52e73eec].


type: ci
user: drh
comment:
  More test cases.  Remove some invalid testcase() macros.  Rearrange some code
  for improved testability.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent trunk changes, include the R-Tree enhancement that allows
  8-byte BLOB arguments to geometry functions, and the fix for the TEXT affinity
  problem that could cause corrupt indexes.


type: ci
user: drh
comment:
  Simplifications to error message processing.  Fix a possible problem in error
  message formatting when vacuuming a database with a corrupt schema.


type: ci
user: dan
comment:
  Do not assume that "col IS ?" matches at most a single row of a UNIQUE column unless the column is also NOT NULL.


type: ci
user: drh
comment:
  Update the README.md file at the top of the repository so that the homepage
  of Git mirrors of SQLite direct users to the official repository.


type: ci
user: drh
comment:
  Fix the transitive constraint processing to only allow transitivity if the
  operands of the == or IS operator have compatible affinities.


type: ci
user: drh
comment:
  Transitive constraints should only work if operands have compatible
  affinities and collating sequences.


type: ci
user: drh
comment:
  Make a hard copy of the results of a subquery lest the result of the
  subquery be referenced after a change to the table that generated the subquery
  result.


type: ci
user: drh
comment:
  Use an ALWAY() on conditionals in the transitive constraint logic that are
  always true as far as we know.


type: ci
user: dan
ecomment:
  Ensure that when the VM applies TEXT affinity to a value it discards any existing REAL or INTEGER value. Fix for [34cd55d6].

comment:
  Ensure that when the VM applies TEXT affinity to a value it discards any existing REAL or INTEGER value.


type: t
user: drh
brief: New ticket [34cd55d68e0e6e7c|34cd55d68e].
comment:
  New ticket [34cd55d68e0e6e7c|34cd55d68e] <i>Database corruption following INSERT with a TRIGGER that does an affinity change</i>.


type: ci
user: drh
comment:
  Allow R-Tree geometry functions to take 8-byte BLOB arguments which are
  passed directly through to the underlying callback, and which can be used
  to pass pointers into the callback.


type: ci
user: drh
comment:
  Ensure that when the VM applies TEXT affinity to a value it discards 
  any existing REAL or INTEGER value. Fix for [34cd55d6].
  Increase the version number to 3.8.10.2.


type: t
user: drh
brief: Fixed ticket [34cd55d68e0e6e7c|34cd55d68e].
comment:
  Fixed ticket [34cd55d68e0e6e7c|34cd55d68e]: <i>Database corruption following INSERT with a TRIGGER that does an affinity change</i> plus 5 other changes


type: t
user: drh
brief: Ticket [34cd55d68e0e6e7c|34cd55d68e]: 3 changes
comment:
  Ticket [34cd55d68e0e6e7c|34cd55d68e] <i>Database corruption following INSERT with a TRIGGER that does an affinity change</i> status still Fixed with 3 other changes


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge the Makefile.msc fix and the compiler warning fix from trunk.  Also
  fix another compiler warning in sqlite3session_diff().


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Add the sqlite3_value_dup() and sqlite3_value_free() interfaces and use those
  to add the sqlite3_rtree_query_info.apSqlParam field to the query callback
  in R-Tree.


type: ci
user: drh
comment:
  Fix handling of queries with VALUES on the left and UNION ALL SELECT on the
  right and a LIMIT clause.


type: ci
user: drh
comment:
  A proposed fix for the problem of CREATE TABLE AS generating a table that
  has INTEGER values in a TEXT column.  Ticket [f2ad7de056ab1dc92].


type: t
user: drh
brief: New ticket [f2ad7de056ab1dc9|f2ad7de056].
comment:
  New ticket [f2ad7de056ab1dc9|f2ad7de056] <i>Corrupt table created by CREATE TABLE AS followed by compound SELECT</i>.


type: t
user: drh
brief: Ticket [f2ad7de056ab1dc9|f2ad7de056]: 6 changes
comment:
  Ticket [f2ad7de056ab1dc9|f2ad7de056] <i>Corrupt table created by CREATE TABLE AS followed by compound SELECT</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  Add a test case to verify that CREATE TABLE AS does not store INT values
  in TEXT columns.  Ticket [f2ad7de056ab1dc92].


type: ci
user: drh
comment:
  Fix the initialization logic in CREATE TABLE AS so that the correct affinities
  are applied to all values being inserted into the new table, even if the
  RHS is a compound SELECT.  Fix for ticket [f2ad7de056ab1dc9200].


type: ci
user: drh
comment:
  Add the sqlite3_value_dup() and sqlite3_value_free() interfaces.  Use
  these interfaces to enhance R-Tree to add the
  sqlite3_rtree_query_info.apSqlParam field.


type: t
user: drh
brief: Fixed ticket [f2ad7de056ab1dc9|f2ad7de056].
comment:
  Fixed ticket [f2ad7de056ab1dc9|f2ad7de056]: <i>Corrupt table created by CREATE TABLE AS followed by compound SELECT</i> plus 3 other changes


type: ci
user: drh
comment:
  Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer.  Cherrypick [ad4b19d2ac0889a]


type: ci
user: dan
comment:
  Fix a problem causing collation sequence names to be dequoted multiple times under some circumstances. Cherrypick of [eddc05e7bb31].


type: ci
user: dan
comment:
  Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. Cherrypick of [ad4b19d2ac0889a2].


type: ci
user: dan
comment:
  Ensure that comparison operators do not mess up the MEM_Dyn flag on registers 
  when reverting affinity changes. Cherrypick of [02e3c88fbf6a].


type: ci
user: dan
comment:
  Avoid signed integer overflow when converting oversized in-line integer 
  widths and precisions in printf(). Cherrypick of [c494171f77dc], [5ce4e7d7651e], [95625ef3adc3] and [8e4ac2ce2441].


type: ci
user: dan
comment:
  Detect and report oversized records constructed from multiple zeroblobs. Cherrypick of [9e139afd9211].


type: ci
user: dan
comment:
  When parsing the schema, ignore any SQL that does not begin with "CREATE". Cherrypick of [d3c00d61581c].


type: ci
user: dan
comment:
  Fix a bug caused by cherrypicking from a branch that assumes sqlite3_stricmp() can handle NULL arguments.


type: ci
user: dan
comment:
  Fix an obscure problem with "INSERT INTO tbl(cols) SELECT" statements where the SELECT is a compound with an ORDER BY and "cols" is a strict subset of tbl's columns. Cherrypick of [718d5d0eab04].


type: ci
user: dan
comment:
  Ensure that tables names are dequoted exactly once by the trigger and FK logic. Cherrypick of [59e92bd9521f] and [9d887b92f808].


type: ci
user: dan
comment:
  Fix a problem causing the fts3 integrity-check to fail if run inside a transaction. Cherrypick of [3b925189a75e].


type: ci
user: dan
comment:
  Avoid ever writing before the start of an allocated buffer in the DIRECT_OVERFLOW_READ code. Fix for [e3a290961a6]. Cherrypick of [c3c15d20c691].


type: ci
bgcolor: #b1d3a8
user: dan
comment: Have r-tree prefer to use the constraint "col MATCH ?" over "rowid = ?".
ecomment:
  Have r-tree prefer to use the constraint "col MATCH ?" over "rowid = ?".
  <i>drh added later:</i>
  All changes on this branch have been cherrypicked to trunk and will be merged
  into sessions from trunk.  So this branch can close.


type: ci
user: drh
comment:
  When parsing the schema, ignore any SQL that does not begin
  with "CREATE". Cherrypick of [d3c00d61581c] with additional changes.


type: ci
user: drh
comment:
  Ensure that tables names are dequoted exactly once by the trigger logic.
  Cherrypick [59e92bd9521f1e8] and [9d887b92f8086961e].


type: ci
user: dan
comment:
  Prevent a virtual table from being destroyed while it is in use. Also: replace Vdbe.inVtabMethod with sqlite3.nVDestroy.  Simplify the EXPLAIN output for P4.pVtab to only show the sqlite3_vtab pointer. Cherrypick of [cbeb9a1aed8c].


type: ci
user: drh
comment:
  The SRT_Table type for the SelectDest object is now just an SRT_EphemTab for
  which the ephemeral table has already been allocated.


type: ci
user: drh
comment:
  Make sure sqlite3_value_dup() works correctly even for sqlite3_value
  objects that are marked SQLITE_STATIC.


type: ci
user: drh
comment:
  Add the sqlite3_value_dup() and sqlite3_value_free() interfaces. 
  Use these interfaces to enhance R-Tree to add the
  sqlite3_rtree_query_info.apSqlParam field.


type: ci
user: drh
comment:
  Fix incorrect validation of the size of the BLOB returned by a geometry
  function on the RHS of a MATCH operator in RTree.  The old code worked for
  64-bit systems (by chance) but fails on 32-bit.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge enhancements from trunk, including the new sqlite3_value_dup() API and
  the addition of the apSqlParam field in the sqlite3_rtree_query_info object
  of R-Tree.


type: ci
user: drh
comment:
  Instead of early detection of corruption in balance_nonroot, simply ignore
  the error, as it is harmless and exceedingly uncommon.


type: ci
user: dan
comment:
  Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions.


type: ci
user: dan
comment:
  Fix a couple of btree asserts that would fail when encountering 32-bit rollover in cell payload size fields (cell payloads this large always indicate corruption).


type: ci
user: dan
comment:
  Have the b-tree layer return SQLITE_CORRUPT to any attempt to open a cursor with a root page number less than 1.


type: ci
user: drh
comment:
  First code for a new utility program to rerun checks on a large number of
  fuzzer-generated test cases.


type: ci
user: dan
comment:
  Fix a case of a corrupt database causing SQLite to read from up to 4 bytes before the start of a memory allocation.


type: ci
user: dan
comment:
  Fix a problem with ignoring UNIQUE constraints on WITHOUT ROWID tables rendered redundant by the PRIMARY KEY.


type: ci
user: dan
comment:
  Return SQLITE_CORRUPT to the user if an attempt is made to add database page 1 to the free page list.


type: ci
user: dan
comment:
  Update test cases in corruptI.test so that they work with both SQLITE_ENABLE_OVERSIZE_CELL_CHECK and SQLITE_DEFAULT_AUTOVACUUM builds.


type: ci
user: dan
comment:
  Add a cast to one side of the test added by [97806a78] in order to avoid a signed/unsigned comparison warning.


type: ci
user: drh
comment:
  Enhance fuzzcheck with the ability to store descriptions in each source
  database and to run multiple source databases in a single invocation.


type: ci
user: drh
comment:
  The "make fuzztest" target now uses fuzzcheck instead of fuzzershell.  Test
  data is added for database fuzz testing.


type: ci
user: dan
comment:
  Avoid branching on an uninitalized variable when comparing SQL values with the undefined serial types 10 and 11 (which only appear in corrupt databases).


type: ci
user: dan
comment:
  Avoid a buffer overread when comparing against a corrupt record that spans at least one overflow page.


type: ci
user: drh
comment:
  Disallow the use of "rowid" in CTEs - it has never worked correctly and it
  makes no sense, so we might as well make it an explicit error.
  Also: add the PRAGMA cell_size_check=ON command.


type: ci
user: drh
comment:
  A different approach to preventing buffer overreads when comparing a vector
  of values with a corrupt index record that spans at least one overflow page.


type: ci
user: dan
comment:
  A different approach to preventing buffer overreads when comparing a vector of values with a corrupt index record that spans at least one overflow page.


type: ci
user: drh
comment:
  Fix a minor and harmless memory leak in fuzzcheck.  Adjust the makefiles
  to run fuzzcheck with --quiet on valgrind tests.


type: ci
user: drh
comment:
  Add the btreeGetUnusedPage() routine to btree.c, and use it to detect content
  pages on the freelist and to cause that condition to trigger an SQLITE_CORRUPT.


type: ci
user: drh
comment:
  Using "SELECT ALL" instead of just "SELECT" on a query that uses a single
  unflattenable subquery or view in its FROM clause will force the subquery to
  be manifested into a temporary table rather than run incrementally using a
  co-routine.  This is a stop-gap means of controlling the decision to manifest
  while we try to work out a better to make that decision automatically.


type: ci
user: dan
comment:
  Add syntax to fts5 used to specify that a phrase or NEAR group should match a subset of columns. For example "[col1 col2 ...] : <phrase>".


type: ci
user: drh
comment:
  Ensure that allocateBtreePage() always clears the MemPage pointer when
  it fails due to an I/O or memory allocation error.


type: ci
user: dan
comment:
  Change fts5 expression processing to avoid linear scans of long doclists caused by phrases that match specific columns only.


type: ci
user: drh
comment:
  Add the built-in affinity() SQL function.  This turns out to be not as useful
  as originally envisioned, so abandon it on a branch.


type: g
user: drh
comment:
  Edit [e3fa8b936b31a487|e3fa8b936b]: Move to branch [/timeline?r=flattener-in-new-file&nd&dp=e3fa8b936b31a487&unhide | flattener-in-new-file]. Mark "Closed".


type: ci
user: drh
comment:
  For FROM-clause subqueries that cannot be flattened, try to push WHERE clause
  terms of the outer query down into the subquery in order to help the subquery
  run faster and/or use less memory.


type: ci
user: dan
comment:
  Reimplement [ec69e09a] so that each call to the xNext() method does not involve two iterations of the match expression tree (only one).


type: ci
user: drh
comment:
  For FROM-clause subqueries that cannot be flattened, try to push relevant
  WHERE clause terms of the outer query down into the subquery in order to help
  the subquery run faster and/or use less memory.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Add the sqlite3changegroup_xxx() APIs to the sessions module. For combining multiple changesets or patchsets.


type: ci
user: drh
comment:
  Add a new opcode to the VDBE that gives the b-tree a hint about the set
  of columns in a table or index that are actually used by the query.


type: ci
user: drh
comment:
  Add the OP_ColumnsUsed opcode (when compiled with
  SQLITE_ENABLE_COLUMN_USED_MASK) as a hint to the b-tree layer as to which
  columns of a btree cursor will be used.


type: ci
user: dan
comment:
  Fix handling of fts5 rowid constraints in the absence of a MATCH clause. Add tests to cover recently added branches.


type: ci
user: drh
comment:
  Provide one final Select tree dump prior to WHERE clause analysis 
  when ".selecttrace 0x400" tracing bit is set with SELECTTRACE_ENABLED.
  Analysis and debug changes only - normal builds are unaffected.


type: ci
user: drh
comment:
  Split out the bulk of the actual VDBE code generation logic from where.c
  into a new file, leaving behind the analysis logic.  This makes the original
  where.c smaller and hopefully easier to edit.


type: ci
user: drh
comment:
  Split more subfunctions of where.c out into a new whereexpr.c source file,
  for improved maintainability.


type: ci
user: drh
comment:
  Split out some source code into new files: wherecode.c, whereexpr.c, and
  treeview.c.  Other minor refactoring changes.


type: ci
user: dan
comment:
  Avoid passing constraints that are unusable due to LEFT or CROSS joins to virtual table xBestIndex() methods.


type: ci
user: dan
comment:
  If a query contains "FROM t1 LEFT JOIN t2, t3, t4", ensure that tables t3 and t4 are not scanned before t2. The trunk already does this.


type: ci
user: drh
comment:
  Resolve FROM-clause subqueries after query planning instead of before.
  Greatly reduce the estimated cost of automatic indexes for VIEWs and
  ephemeral tables since performance problems there cannot be mitigated
  via a CREATE INDEX.


type: ci
user: drh
comment:
  Code refactoring to try to shift FROM-clause subquery manifesting until after
  the query planner runs.  Except this does not currently work because the
  query planner needs an estimated of the number of rows in the manifested table.
  Work in progress.


type: ci
user: drh
comment:
  "test" targets on all makefiles use --verbose=file --output=test-out.txt.
  Add the new "quicktest" target to all makefiles - designed to run in under
  three minutes.  The --quick option on releasetest.tcl now uses quicktest.


type: ci
user: drh
comment:
  When generating code for partial indexes, be sure not to modify the
  index condition expression in the schema.


type: ci
user: drh
comment:
  Add a fast-path implementation of pcache1Fetch() for the common case of
  separate caches that do not use a mutex.


type: ci
user: drh
comment:
  Ensure that the CREATE TABLE AS statement correctly undoes partial changes
  to the sqlite_master table if the SELECT on the right-hand side aborts with
  an error.  Fix for ticket [873cae2b6e25b]


type: t
user: drh
brief: Fixed ticket [873cae2b6e25b199|873cae2b6e].
comment:
  Fixed ticket [873cae2b6e25b199|873cae2b6e]: <i>Schema corruption after aborted CREATE TABLE AS</i> plus 5 other changes


type: ci
user: drh
comment:
  Make getCellInfo() a real function instead of a macro, for a size reduction
  and a 0.2% performance gain.


type: ci
user: drh
comment:
  Fix a uninitialized variable use in the command-line shell when the
  ".open" command is invoked without any arguments.


type: ci
user: drh
comment:
  Split out all four cases of cellSizePtr() into different methods, each optimized
  for the specific page type.  This gives a 0.36% performance increase at a cost
  of about 300 bytes of code.


type: ci
user: drh
comment:
  Add #ifdefs to some features speedtest1.c so that it compiles with 
  older versions of SQLite, going back to version 3.5.1.


type: ci
user: drh
comment:
  Performance improvement in sqlite3VdbeMemShallowCopy() for the common
  case where the receiver does not require prior resource deallocation.


type: ci
user: drh
comment:
  Pervent the likelihood() function from moving ON clause terms of a LEFT JOIN
  into the WHERE clause.
  Fix for ticket [5f60b11fc8e23490e2]


type: t
user: drh
brief: New ticket [5f60b11fc8e23490|5f60b11fc8].
comment:
  New ticket [5f60b11fc8e23490|5f60b11fc8] <i>likely() function in the ON clause of a LEFT JOIN results in incorrect result</i>.


type: ci
user: drh
comment:
  Further #ifdef changes in speedtest1.c in order to support SQLite back to
  version 3.3.9 and perhaps even earlier.


type: t
user: drh
brief: Fixed ticket [5f60b11fc8e23490|5f60b11fc8].
comment:
  Fixed ticket [5f60b11fc8e23490|5f60b11fc8]: <i>likely() function in the ON clause of a LEFT JOIN results in incorrect result</i> plus 5 other changes


type: ci
user: drh
comment:
  Make cellSizePtr() a method on the MemPage object, with alternative
  implementations depending on the page type.  This results is a small performance
  improvement and size reduction.


type: ci
user: drh
comment:
  Add the MemPage.xParseCell method and provide various implementations
  (variations on the former btreeParseCellPtr()) depending on the page type.


type: ci
user: drh
comment:
  Make sure that the likelihood() function does not mess up generation of
  result-set code when it is used (inappropropriately) in a result-set 
  expression.


type: ci
user: drh
comment:
  Cache the most recently sqlite3_context used by OP_Function and reuse it on
  subsequent calls, if appropriate. This gives a noticable performance boost.


type: ci
user: drh
comment:
  Avoid computing cell sizes in balance_nonroot() until they are really needed.
  This gives an overall 1.7% performance gain for about 1000 extra bytes of
  code space.


type: ci
user: drh
comment:
  Change the way that balance_nonroot() partitions cells between the sibling
  pages such that a scan of the cell size array is not required.


type: ci
user: dan
comment:
  Test that the left and right sides of a compound SELECT operator have the same  number of expressions in the expanded expression list before beginning to generate code.


type: ci
user: drh
comment:
  Change pageInsertArray() and pageFreeArray() so that they use the CellArray
  object and compute cell sizes as needed, resulting in smaller and faster code.


type: ci
user: drh
comment:
  Avoid unnecessary cachedCellSize() calls in the cell partition adjustment
  phase of balance_nonroot().


type: ci
user: drh
comment:
  Multiple overflow cells are always adjacent and sequential.  Exploit this
  invariant for a small size reduction and performance increase and add
  assert()s to prove the invariant.


type: ci
user: dan
comment:
  Add a script to combine all fts5 code into a single file - fts5.c - that can be used to build an SQLite loadable extension.


type: ci
user: drh
comment:
  Performance optimization on balance_nonroot() and related routines.
  2.6% faster overall with a size increase of less than 750 bytes.


type: ci
user: drh
comment:
  Mark some branches as unreachable after the recent change that recognizes
  mismatch result set sizes on compound SELECT statements sooner.


type: ci
user: drh
comment:
  Adjustment to a malloc test so that it accepts a narrow range of values to
  account for variations in malloc subsystems.


type: ci
user: drh
comment:
  Rig the fuzzcheck test program so that it times out after 10 seconds in case
  of an infinite loop in the test case.


type: ci
user: drh
comment:
  Fix the progress handler so that it does not fire too soon during an
  sqlite3_step() call other than the first.


type: ci
user: drh
comment:
  In the fuzzcheck test program, use the progress handler to limit the number
  of VDBE cycles to avoid getting stuck if the SQL under test contains an
  infinite CTE loop.  Add the --limit-vdbe command-line option.


type: ci
user: drh
comment:
  Increase the default timeout on fuzzcheck to 120 seconds - sufficient for
  running under valgrind on a slow workstation.


type: ci
user: drh
comment:
  Enhance the Btree object to remember whether or not it is holding an
  Incrblob cursor.  Use this knowledge to improve performance in the common
  case where it does not.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge all the latest enhancements from trunk.  This merge include FTS5
  and a number of notable performance enhancements.


type: ci
user: drh
comment:
  More simplifications and performance improvements to cell allocation
  logic associated with allocateSpace().


type: ci
user: dan
comment:
  Fix Makefile.in to account for recent changes. Update fts5 test scripts so that they do not run if SQLITE_ENABLE_FTS5 is not defined.


type: ci
user: drh
comment:
  Simplify the pcache by not keeping continuous track of page 1 but instead
  just loading page 1 on the rare occasions when it is actually needed.


type: ci
user: drh
comment:
  Optimization to sqlite3_value_bytes() for the common case that does not
  require a data transformation.


type: ci
user: drh
comment:
  Cache the sqlite3_context structure in the P4 operand of VDBE programs
  for faster SQL function dispatch.


type: ci
user: drh
comment:
  Rework the PRAGMA integrity_check logic.  Simplify the checkTreePage() routine
  and clean up the error messages generated.


type: ci
user: drh
comment:
  Add the Mempage.aDataOfst field and use it in sqlite3BtreeMovetoUnpacked()
  for about a 2 million cycle gain.


type: ci
user: drh
comment:
  Add releasePageNotNull() as an alternative to releasePage() when the argument
  is guaranteed to be non-NULL, and use it for a small performance gain.


type: ci
user: drh
comment:
  Manually inline the call from getAndInitPage() to btreeGetPage() for a
  savings of 2.5 million cycles at a cost of less than 100 bytes.


type: ci
user: drh
comment:
  Performance improvements in moveToChild() by shifting some work over
  to getAndInitPage().  Net improvement is about 800K cycles at cost of 30 bytes.


type: ci
user: drh
comment:
  Avoid unnecessary calls to sqlite3VdbeSerialTypeLen() for integer
  serial types, for a small size reduction and a speed increase.


type: ci
user: drh
comment:
  Add the new PGHDR_CLEAN bit to PgHdr.flags in pcache.c.  This bit is always
  the opposite of PGHDR_DIRTY.  Use the extra bit to avoid a comparison
  for a small performance boost.


type: ci
user: drh
comment:
  Add the PGHDR_WRITEABLE bit for PgHdr.flags which is used to 
  distinguish between pages that are on the dirty list and pages that are
  safe to modify.


type: ci
user: drh
comment:
  Add a bypass path in sqlite3PagerWrite() for pages with the PGHDR_WRITEABLE
  bit set, for about a 1% performance increase.


type: ci
user: drh
comment:
  From sqlite3BtreeInsert(), bypass the btreeMoveto() routine for table inserts
  and go directly to sqlite3BtreeMovetoUnpacked().


type: ci
user: drh
comment:
  Put BtCursor objects on a singly-linked list instead of a doubly-linked list.
  Add the BTCF_Multiple flag.  Only invoke saveAllCursors() when using a cursor
  that has BTCF_Multiple set.


type: ci
user: drh
comment:
  Put all variable declarations for sqlite3PagerWrite()
  at the beginning of the block.  Syntax only - no logic changes.


type: ci
user: drh
comment:
  Change an unreachable branch into an assert() in sqlite3PagerAcquire()
  and optimize sqlite3PcacheOpenSavepoint() by factoring out rarely used code
  into a subroutine.


type: ci
user: drh
comment:
  Make use of htonl() and <nowiki>__builtin_bswap32()</nowiki> for faster
  implementations of sqlite3Get4byte() and sqlite3Put4byte().


type: ci
user: drh
comment:
  Remove the use of htonl() in the previous check-in due to linkage issues.
  Add the get2byteAligned() macro and use it for access to the cell offsets
  on btree pages for about a 1% performance gain.


type: g
user: drh
comment:
  Edit [bc27ebd7f73e9fc8|bc27ebd7f7]: Move to branch [/timeline?r=bswap-functions&nd&dp=bc27ebd7f73e9fc8&unhide | bswap-functions].


type: ci
user: drh
comment:
  Make use of built-in bswap32() and bswap16() functions in GCC/Clang for a
  significant performance improvement there.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge in the latest enhancements from trunks, especially the use of
  _byteswap_ulong() and similar intrinsics on MSVC.


type: ci
user: drh
comment:
  Change integrity_check to analyze the cells of a page in reverse order, as 
  this will tend to insert the cells into the analysis heap in increasing order
  by address, which is the most efficient way to load a min-heap.


type: ci
user: mistachkin
comment:
  Make use of the _byteswap_ushort() and _byteswap_ulong() compiler intrinsics for MSVC, when available.


type: ci
user: dan
comment:
  Within a write-transaction on a wal database in mmap mode, check the page-cache for an entry before the wal file. This can be faster if the wal file is very large.


type: ci
user: drh
comment:
  Avoid create a stack from in sqlite3BackupUpdate() in the common case where
  the first argument is NULL.


type: ci
user: drh
comment:
  Simplification of PRAGMA integrity_check logic.  Make sure that the depth
  of the right-most subtree is correct.  Size reduction and performance increase,
  with no change in output.


type: ci
user: dan
comment:
  Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c.


type: ci
user: drh
comment:
  Smaller and faster PRAGMA integrity_check that also does a better job of
  detecting errors.  Some output text describing discovered file corruption
  has changed for clarity.


type: ci
user: dan
comment:
  Minor change to mkfts5c.tcl so that fts5.c can be used according to the instructions in loadext.html.


type: ci
user: drh
comment:
  When doing coverage analysis of a btree page for pragma integrity_check,
  make the first entry (that covers the header, cell index, and gap) implied,
  for a performance boost and size reduction.


type: ci
user: dan
comment:
  Copy fts5.h into the build directory as part of building fts5.c. Also remove code to edit fts5parse.c from various Makefiles and add it to mkfts5c.tcl.


type: ci
user: drh
comment:
  New command-line options for speedtest1: --multithread, --nomemstat,
  --serialized, and --singlethread


type: ci
user: mistachkin
comment:
  Add static mutexes for use by the built-in / third-party VFSs and use the built-in VFS mutex where appropriate.


type: ci
user: drh
comment:
  Do not preserve the btreeInitPage() results in PRAGMA integrity_check if
  there were cell sizing errors because PRAGMA cell_size_check was turned off
  during the btreeInitPage() call.


type: ci
user: drh
comment:
  Preserve the number of requested PAGECACHE pages even if the memory pointer
  or size is zero.  Enhance the pcache1.c header comment to explain the memory
  layout of a page cache line.


type: ci
user: drh
comment:
  Enhance separate pcache1 to allocate a block of pages from heap on startup,
  if possible, for a 5.2% performance improvement.


type: ci
user: drh
comment:
  A unified cache is now only used for SQLITE_ENABLE_MEMORY_MANAGEMENT, or if
  SQLITE_CONFIG_PAGECACHE defines a shared start-time page cache buffer and the
  application is single-threaded.


type: ci
user: dan
comment:
  Add a test case to verify that "PRAGMA data_version" works as expected when an OTA client writes to the database.


type: ci
user: drh
comment:
  Early detection of file corruption in a debugging routine, before the invalid
  data can cause problems.


type: ci
user: drh
comment:
  Avoid left-shifting bits into the sign of a signed integer, as such behavior
  is technically undefined.


type: ci
user: drh
comment:
  Do not do the bulk pcache1 allocations if SQLITE_CONFIG_PAGECACHE specifies
  a global page cache buffer.


type: ci
user: drh
comment:
  Enhance the pcache1 page cache so that it tries to allocate a block of
  SQLITE_DEFAULT_PCACHE_INITSZ pages from malloc() on startup, and uses those
  preallocated pages when possible rather than going to malloc() for each
  individual page.  About a 5% performance increase for some workloads.


type: ci
user: dan
comment:
  Fix two problems that could cause fts3 auxiliary functions to occasionally misbehave if used with match expressions that contain both OR and NEAR.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge all changes in the latest 3.8.11 beta into the sessions branch.
  Changes include the rename of OTA to RBU, the WITHOUT-ROWID-OR-Skipscan fix,
  and improvements to pcache1.


type: ci
user: drh
comment:
  Amplify the comment on renderLogMsg() that explains the problems associated
  with calling sqlite3_log() from deep within the memory allocator.


type: ci
user: dan
comment:
  Use a WITHOUT ROWID table to index fts5 btree leaves. This is faster to query and only slightly larger than storing btree nodes within an intkey table.


type: ci
user: drh
comment:
  Defer the bulk pcache1 memory allocation until the first page allocation
  request.  Limit the size of the pcache1 bulk allocation to the cache_size
  setting.  Deallocate the bulk allocation on a sqlite3_db_release_memory()
  request, if the bulk allocation is completely unused.


type: ci
user: drh
comment:
  More adjustments to the malloc5.test module so that it works correctly with
  the new pcache1 bulk allocation.


type: ci
user: drh
comment:
  The sqlite3ExprCodeGetColumn() is not guaranteed to put the result in the
  register requested.   Fix the skip-scan code generator for WITHOUT ROWID
  tables so that it always checks the register and copies the result if it
  lands in the wrong register.  Fix for ticket [8fd39115d8f46ece70e7d4b3].


type: t
user: drh
brief: New ticket [8fd39115d8f46ece|8fd39115d8].
comment:
  New ticket [8fd39115d8f46ece|8fd39115d8] <i>assertion fault on WITHOUT ROWID query using the OR optimization</i>.


type: ci
user: drh
comment:
  When SQLITE_OMIT_LOOKASIDE is set, do not allocate any lookaside memory since
  it will never be used.


type: t
user: drh
brief: Fixed ticket [8fd39115d8f46ece|8fd39115d8].
comment:
  Fixed ticket [8fd39115d8f46ece|8fd39115d8]: <i>assertion fault on WITHOUT ROWID query using the OR optimization</i> plus 7 other changes


type: ci
user: drh
comment:
  More robust handling of zeroblob() with oversized arguments.  Fix fuzzcheck
  so that it can be run with limited heap memory.


type: ci
user: drh
comment:
  New makefile target "fastfuzztest" runs the same tests as "fuzztest" but
  with a 100M memory size limit.  This more than doubles the speed.  The
  original unlimited "fuzztest" is still run on a "fulltest".


type: ci
user: dan
comment:
  Allow the RBU module to read data from appropriately named SQL views created within the RBU database.


type: ci
user: drh
comment:
  Fix the zeroblob.test script so that it works even on builds where the
  SQLITE_MAX_LENGTH has been raised to 2147483645.


type: ci
user: dan
comment:
  Remove a potentially undefined behaviour involving signed integers and bitshift operations from fts5_hash.c.


type: ci
user: drh
comment:
  Fix the xfer optimization for WITHOUT ROWID tables so that it correctly
  updates the change count returned by sqlite3_changes().


type: ci
user: dan
comment:
  Only allow UNLOCKED transactions to commit if none of the pages read by the transaction have been modified since it was opened.


type: ci
user: dan
comment:
  Handle writes to auto-vacuum databases within UNLOCKED transactions in the same way as for non-UNLOCKED transactions.


type: ci
user: drh
ecomment:
  The "PRAGMA cache_size" reads the database schema, as it has done in all prior
  versions of SQLite.  This side-effect of cache_size is undocumented, but 
  apparently it does get used and its absence from 3.8.11 was causing problems.

comment:
  The "PRAGMA cache_size" reads the database schema, as it has down in all prior
  versions of SQLite.  This side-effect of cache_size is undocumented, but 
  apparently it does get used and its absence from 3.8.11 was causing problems.


type: ci
user: drh
comment:
  Restore an undocumented side-effect of PRAGMA cache_size: it forces the
  database schema to be parsed.  Also increase the version number to 3.8.11.1.


type: ci
user: drh
comment:
  Fix the releasetest.tcl script so that it ignore cast overflow errors that
  recent versions of clang generate but which are unavoidable given the test
  input.


type: ci
user: mistachkin
comment:
  Add SQLITE_DISABLE_INTRINSIC define to disable use of intrinsic functions (e.g. for certain older compilers and/or platforms).


type: ci
user: dan
comment:
  Return an error message (instead of segfaulting) if test function fts5_expr() is invoked with no arguments.


type: ci
user: dan
comment:
  Allow RBU tables to be named "data[0-9]*_<target>" instead of strictly "data_<target>". Also update RBU so that it always processes data tables in order sorted by name.


type: ci
user: dan
comment:
  Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer.


type: ci
user: dan
comment:
  Fix a problem causing [sqldiff --rbu] to fail on tables for which all columns are part of the PRIMARY KEY.


type: ci
user: mistachkin
comment:
  When building the VSIX package for UAP, omit the AppLocal suffix for the MSVC runtime library because it is no longer needed.


type: ci
user: dan
comment:
  Remove a duplicate "typedef struct Fts5Global Fts5Global;" from fts5_main.c (duplicate is in fts5Int.h).


type: ci
user: drh
comment:
  Fix the sqlite3_stmt_busy() interface so that it always returns FALSE after
  the statement has returned SQLITE_DONE, even for ROLLBACK statements.
  Clarify the documentation.


type: ci
user: drh
comment:
  Allow a VIEW to reference undefined tables and functions when initially
  created.  The error report is deferred until the VIEW is used.  This allows
  views to be created before subviews and tables that the view references.


type: ci
user: dan
comment:
  Add the "--uri" option to utility program "showdb". This option causes showdb to use sqlite with the SQLITE_OPEN_URI option to open the database file, instead of opening it directly using the OS open() function.


type: ci
user: dan
comment:
  Update the [showdb] program so that the database file is read directly (bypassing the SQLite VFS) only if the --raw option is specified. Otherwise, it is read using the default VFS. An alternative VFS may be specified by specifying the database file to open using the URI syntax.


type: ci
user: dan
comment:
  Update the [showdb] program so that the database file is read directly (bypassing the SQLite VFS) only if the --raw option is specified. Otherwise, it is read using the default VFS. Also, the URI syntax may be used on the command line to specify the name of the database file to examine, so an alternative VFS may be requested using a URI parameter.


type: ci
user: drh
comment:
  In the sqlite3_analyzer.exe utility, show the depth of each btree and report
  the average fanout of indexes and WITHOUT ROWID tables.


type: ci
user: dan
comment:
  Add new test file fts5_test_mi.c, containing an implementation of a function similar to FTS4 matchinfo() for FTS5.


type: ci
user: dan
comment:
  Update the spellfix virtual table extension so that an explicit "top = ?" constraint works even if there is also a "distance < ?" or "distance <= ?" constraint.


type: ci
user: dan
comment:
  Add tests for the matchinfo-like test function. Fix problems found in test and fts5 code by doing so.


type: ci
user: drh
comment:
  Enhance the query planner so that it is able to use partial indexes that use
  AND-connected terms in the WHERE clause.


type: ci
user: dan
comment:
  Add a specially formatted comment to shell.c to make it easier for scripts to edit. No code changes.


type: g
user: drh
comment:
  Edit [be190fe7826d157f|be190fe782]: Move to branch [/timeline?r=warnings&nd&dp=be190fe7826d157f&unhide | warnings].


type: ci
user: drh
comment:
  Add the sqlite3_experimental_log_open() interface.  This is for diagnostic
  use only and is not intended to ever go on trunk.


type: ci
user: dan
comment:
  Add the xPhraseFirst() and xPhraseNext() fts5 APIs, for faster iteration through a single phrases position list. Also optimize xInst() and xInstCount() a bit.


type: ci
user: drh
comment:
  Merge trunk enhancements into the apple-osx branch.  Most tests works, but
  there are yet a few issues to be resolved.


type: ci
user: dan
comment:
  When searching the wal file for a frame, do not search that part that was already checkpointed when the snapshot being read was at the head of the wal file.


type: ci
user: dan
comment:
  Add the sqlite3rbu_savestate() function to the rbu extension. To force rbu to save its state to disk without closing the sqlite3rbu* handle.


type: ci
user: dan
comment:
  Avoid invoking system call unlink() directly from RBU code. Use the xDelete method of the default VFS instead.


type: ci
user: drh
comment:
  Merge in all the trunk changes from the previous year.  This breaks the
  cursor-hint mechanism, but provides a baseline for trouble-shooting.


type: ci
user: drh
comment:
  Fix the CursorHint so that it includes the scan boundary constraints.
  On the expression text for the CursorHint opcode, show rowid correctly.


type: ci
user: drh
comment: Small performance gain and size reduction in sqlite3VdbeIdxKeyCompare().
ecomment:
  This change is not correct for the boundary case of nCellKey==0.  Was:
  Small performance gain and size reduction in sqlite3VdbeIdxKeyCompare().


type: ci
user: drh
comment:
  Minor fix to the display of oversized KeyInfo structures in the P4 operand
  of Open opcodes in EXPLAIN output.


type: ci
user: drh
comment:
  Updated testing of cursor-hints.  Remove the test_cursorhint.c file and
  associated logic in the core and do tests based purely on the newly enhanced
  EXPLAIN output.


type: ci
user: drh
comment:
  Always provide the BTREE_BULKLOAD hint, even when SQLITE_ENABLE_CURSOR_HINTS
  is not defined, as that hint gives a 4% performance increase.


type: ci
user: drh
comment:
  Avoid generating hints using constraints that are also used to initialize the
  cursor, since presumably the cursor already knows about those constraints.


type: ci
user: dan
comment:
  When committing an unlocked transaction, relocate newly allocated database pages within the file to avoid conflicting with committed transactions. There are lots of things still to fix in this code.


type: ci
user: drh
comment:
  Abandon the JSONB format for now.  (We may return to it in the future.)  Add
  a function to render a JSON parse.


type: ci
user: drh
comment:
  Provide hints for all terms in a range constraint if there are any equality
  terms anywhere in the constraint.  Range constraint terms are only omitted
  for a pure range constraint with no equality prefix.


type: ci
user: dan
comment:
  When updating a zipvfs database with RBU, set journal_mode=off to prevent the upper-level pager from creating a large temporary file.


type: ci
user: drh
comment:
  Virtual table modules with a null xCreate method act as eponymous-only modules -
  they cannot be used in a CREATE VIRTUAL TABLE statement.  Add the series.c
  extension that implements a postgres-like generate_series virtual table to
  demonstrate this capability.


type: g
user: drh
comment:
  Edit [c1f43a7799a9298a|c1f43a7799]: Move to branch [/timeline?r=table-valued-functions&nd&dp=c1f43a7799a9298a&unhide | table-valued-functions].


type: ci
user: drh
comment:
  Minor refactor of the SrcList object so that it is able to hold the argument
  list to a table-valued-function in the FROM clause.


type: ci
user: drh
comment:
  A list of arguments following a table name translates into equality
  constraints against hidden columns in that table.


type: ci
user: drh
comment:
  Improved comments on the generate_series virtual table.  Test cases for
  ORDER BY rowid DESC with generate_series.


type: ci
user: drh
comment:
  Fix eponymous virtual tables so that they do not automatically make the first
  column the rowid.  Enhance the generate_series virtual table to support rowid.


type: ci
user: drh
comment:
  Fix the path lookup for objects so that it can handle quoted identifier
  names and non-alphanumerics in the identifier.


type: ci
user: drh
comment:
  Prevent the series.c extension from loading on older versions of SQLite that
  do not support xCreate==NULL.


type: ci
user: drh
comment:
  Fix corner-case memory management issues in table-valued functions.  Change
  virtual table handling so that if xDestroy is missing the table is
  eponymous only even if xCreate is present.


type: ci
user: drh
comment:
  Fix a corner-case bug in table-valued functions.  Update the generate_series()
  virtual table to increase the performance estimate penalty for being
  underspecified.


type: ci
user: dan
comment:
  When committing an UNLOCKED transaction, try to move pages allocated at the end of the file to free slots within the file (like an incremental-vacuum operation does).


type: ci
user: drh
comment:
  Change the name of the json loadable extension to "json1.c", in anticipation
  of having future major changes to the interface.


type: ci
user: drh
comment:
  Fixes for compiler warnings and errors in the makefiles.  Rename the
  one test script to json101.test.


type: ci
user: drh
ecomment:
  Do not apply the WHERE-clause pushdown optimization to terms that originate
  in the ON or USING clause of a LEFT JOIN.  Fix for ticket
  [c2a19d81652f40568c].

comment:
  Do not apply the WHERE-clause pushdown optimization to terms that originate
  in the ON or USING clause of a LEFT JOIN.  Fix for ticket
  [6df18e949d3676290].


type: t
user: drh
brief: New ticket [c2a19d81652f4056|c2a19d8165].
comment:
  New ticket [c2a19d81652f4056|c2a19d8165] <i>Incorrect LEFT JOIN when FROM clause contains nested subqueries</i>.


type: t
user: drh
brief: Fixed ticket [c2a19d81652f4056|c2a19d8165].
comment:
  Fixed ticket [c2a19d81652f4056|c2a19d8165]: <i>Incorrect LEFT JOIN when FROM clause contains nested subqueries</i> plus 5 other changes


type: ci
user: dan
comment:
  Fix a problem to do with detecting unlocked transaction conflicts if another client restarts the wal while the transaction is running.


type: ci
user: drh
comment:
  Add the json_valid() function to the json1.c extension.  Fix various minor
  problems in the json1.c extension.


type: ci
user: drh
comment:
  Disallow the use of COLLATE clauses and the ASC and DESC keywords within
  foreign key constraints and in the argument list to common table expressions.


type: ci
user: dan
comment:
  Fix handling of attempts to modify the database schema, application_id or user_version within an UNLOCKED transaction.


type: ci
user: drh
comment:
  Enhance the CREATE VIEW syntax so that the names of columns of the view can
  be specified after the view name.


type: ci
user: drh
comment:
  Enhances the parser so that it accepts arbitrary expressions for the arguments
  of an index, though the code generator still rejects everything other than
  simple column names.  The sqlite3RestrictColumnListSyntax() routine is removed
  since that feature is now handled by the parser.


type: ci
user: drh
comment:
  Remove some redundant code:  Call sqlite3ResolveExprListNames() rather than
  calling sqlite3ResolveExprNames() in a loop - in two places.


type: ci
user: drh
ecomment:
  Changes toward being able to process indexes on expressions.  Not there yet - 
  this check-in is just movement in that direction.  Some tests are failing.

comment:
  Changes toward being abld to process indexes on expressions.  Not there yet - 
  this check-in is just movement in that direction.  Some tests are failing.


type: ci
user: drh
comment:
  Add code to maintain indexes with expression arguments across DELETE, INSERT,
  and UPDATE statements.  Legacy tests pass, but the new code paths are still
  largely untested.  The query planner currently makes no effort to use
  expression indexes.


type: ci
user: dan
comment:
  Test that if a corrupt wal-index header is encountered when attempting to commit a concurrent transaction, SQLITE_BUSY_SNAPSHOT is returned to the caller.


type: ci
user: dan
comment:
  If "PRAGMA integrity_check" is run while the database is being written by a CONCURRENT transaction, do not consider unreferenced pages to be an error. They may be part of the free-page list, which is not visible at the b-tree layer when running a CONCURRENT transaction.


type: ci
user: drh
comment:
  Use the sqlite3IndexColumnAffinity() routine to quickly and correctly find the
  affinity of an index column.


type: ci
user: drh
comment:
  Update the TreeView output for compound SELECT statements so that all entries
  are shown vertically aligned rather than each successive entry being indented.


type: ci
user: drh
comment:
  Evaluate expressions only once when the same expression is used in both the
  result set and in the ORDER BY clause.


type: ci
user: dan
comment:
  Fix a problem whereby concurrent transactions would not consider pages read by the transaction before the first write statement.


type: ci
user: drh
comment:
  Activate the ability to use expressions in indexes in a query.  There are some
  test failures, but mostly this seems to work.


type: ci
user: drh
comment:
  Fix EXPLAIN QUERY PLAN output for indexed-expressions.  Fix another
  obscure fault in the WHERE term scanner.


type: ci
user: drh
comment:
  Disable the read-only WAL-mode database tests on the apple-osx branch because
  read-only WAL-mode databases are specifically disallowed by Apple-specific
  changes.


type: ci
user: drh
comment:
  Add the json_check() function, which returns its argument if the argument
  is well-formed JSON or which throws an error otherwise.


type: ci
user: drh
comment:
  Enhance the json_insert(), json_replace(), and json_set() functions with the
  ability to add JSON instead of text if the argument is text and if the PATH
  begins with '$$' instead of just '$'.


type: ci
user: dan
comment:
  When searching the wal file for a frame, do not search that part that was already checkpointed when the transaction was opened.


type: ci
user: dan
comment:
  Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document.


type: ci
user: drh
comment:
  Change the json1.c module so that it throws an error if any of the
  JSON selector paths are malformed.


type: ci
user: drh
comment:
  Back out the json_check() routine.  Instead, throw an error if the input to
  a json function (other than json_valid()) is not valid JSON.


type: ci
user: dan
comment:
  Add a test for an fts5 tokenizer that supports synonyms by adding multiple entries to the fts index.


type: ci
user: dan
comment:
  Begin changes to allow synonym support by adding multiple terms to a query (an alternative to adding multiple terms to the FTS index).


type: ci
user: drh
comment:
  Enhance the json_extract() function so that if given multiple PATH arguments
  it will return a JSON array with all of the answers.  Also update comments
  within the json1 extension to reflect stricter interpretation of JSON and PATH
  arguments.


type: ci
user: drh
comment:
  Make the distinction between truly deterministic functions and date/time
  functions which only return the same answer for a single query.  Only truly
  deterministic functions are allowed in indexes.  Add new expression index
  test cases.


type: ci
user: drh
comment:
  Always assume that indexed expressions can generate a NULL.  Get
  indexed expressions working for the case of two or more expressions in
  the same index.


type: ci
user: drh
comment:
  Not only date/time functions, but also functions like sqlite_version() and
  changes() need to be prohibited from use inside of indexes.


type: ci
user: drh
comment:
  Rename SQLITE_FUNC_VARYING to SQLITE_FUNC_SLOCHNG - a more descriptive name
  for what that bit means.


type: g
user: drh
comment:
  Edit [8079421604dbd40d|8079421604]: Move to branch [/timeline?r=begin-concurrent&nd&dp=8079421604dbd40d&unhide | begin-concurrent].


type: ci
user: drh
comment:
  Use sqlite3XPrintf() instead of sqlite3StrAccumAppend() in a few places
  for better performance and a smaller footprint.


type: ci
user: drh
comment:
  The sqlite3_memory_alarm() interface has been deprecated and undocumented
  for almost 8 years (since version 3.5.3).  Change it into a no-op.


type: g
user: drh
comment:
  Edit [6e0298cdcb3b900f|6e0298cdcb]: Move to branch [/timeline?r=mistake&nd&dp=6e0298cdcb3b900f&unhide | mistake]. Edit check-in comment.


type: ci
user: drh
comment:
  Change sqlite3_sql() so that it always returns the SQL text of the prepared
  statement, if the text is available, even if the the prepared statement was
  generated using sqlite3_prepare() instead of sqlite3_prepare_v2().  Also
  include some minor comment fixes and code simplifications.


type: ci
user: drh
comment:
  Add and use the sqlite3VdbeChangeOpcode() routine.  Simplify the implementation
  of the other sqlite3VdbeChange*() routines.


type: ci
user: drh
comment:
  Factor out and simplify code in pragma.c for pragmas that return a single
  row with a single column that is an integer or text value.


type: ci
user: drh
comment:
  Add the sqlite3VdbeLoadString() and sqlite3VdbeMultiLoad() routines to help
  with code generation, especially on PRAGMAs.  Rename sqlite3VdbeAddGoto()
  to just sqlite3VdbeGoto().


type: ci
user: drh
comment:
  Changes from ENABLE_CONCURRENT (default off) to OMIT_CONCURRENT (default on).
  This is not a clear-cut decision and might be changed back.


type: ci
user: drh
comment:
  Merge trunk enhancements, and espeically the fix for allowing strings 
  as column identifers in CREATE INDEX statements.


type: ci
user: drh
comment:
  Merge performance enhancements from trunk.  This branch now runs (slightly)
  faster than the 3.8.11.1 release, though still slightly slower than trunk.


type: ci
user: drh
comment:
  Change the Pager.hasBeenUsed flag into Pager.hasHeldSharedLock in order to
  take it off the critical path in sqlite3PagerAcquire().


type: ci
user: drh
comment:
  Change the pcache module to keep track of the total number of references to
  all pages rather than the number of pages references, for a performance
  improvement and size reduction.


type: ci
user: drh
comment:
  Remove the EXCLUSIVE and CONCURRENT tokens from the tokenizer.  Let the
  BEGIN statement be followed by an ID, but throw a syntax error if the ID
  is anything other than EXCLUSIVE or CONCURRENT.


type: ci
user: dan
comment:
  Modify the fts5 custom tokenizer interface to permit synonym support. The fts5_api.iVersion value is now set to 2. Existing fts5 custom tokenizers (if there are such things) will need to be updated to use the new api version.


type: ci
user: drh
comment:
  Continue to support the (broken) legacy syntax of allowing strings for
  column names in CREATE INDEX statements and in UNIQUE and PRIMARY KEY
  constraints.


type: ci
user: drh
comment:
  Add support for CREATE INDEX statements that use deterministic expressions
  rather than only column names.


type: ci
user: drh
comment:
  Merge the latest trunk changes, and especially the fix for allowing
  strings as identifiers in CREATE INDEX statements.


type: ci
user: drh
comment:
  Add a memory barrier to the mutex initialization logic, try to work around
  an issue reported by WebKit.


type: ci
user: drh
comment:
  Omit all use of Expr nodes for TK_AS, as those nodes no longer served a useful
  purpose and in fact interferred with the query planner.


type: ci
user: drh
comment:
  In the "parse.out" output file from Lemon, show addition the complete text
  of rules on reduce actions.


type: ci
user: drh
comment:
  For the Lemon-generated parser, add a new action type SHIFTREDUCE and use it
  to further compress the parser tables and improve parser performance.


type: ci
user: drh
comment:
  Change the parser engine so that it (once again) waits for a lookahead token
  before reducing, even in a SHIFTREDUCE action.


type: g
user: drh
comment:
  Edit [531c3974b3d586c1|531c3974b3]: Move to branch [/timeline?r=lemon-update&nd&dp=531c3974b3d586c1&unhide | lemon-update].


type: ci
user: drh
comment:
  Enhance the Lemon parser generator to add SHIFTREDUCE states that reduce the
  sizes of some of the parser tables.


type: ci
user: drh
comment:
  Enhance the DBSTAT virtual table with a new hidden table "schema" that if
  set will cause the table to report on the specified schema rather than on
  "main".  Also:  Fix a faulty assert in sqlite3_context_db_handle().


type: ci
user: drh
comment:
  When running a CREATE TABLE AS, make the initial temporary sqlite_master entry 
  for the new table a real record rather than a NULL, in case the query after 
  the AS actually tries to read the sqlite_master table.
  Fix for ticket [acd12990885d9276].


type: t
user: drh
brief: New ticket [acd12990885d9276|acd1299088].
comment:
  New ticket [acd12990885d9276|acd1299088] <i>CREATE TABLE AS fails with a false report of database corruption</i>.


type: t
user: drh
brief: Fixed ticket [acd12990885d9276|acd1299088].
comment:
  Fixed ticket [acd12990885d9276|acd1299088]: <i>CREATE TABLE AS fails with a false report of database corruption</i> plus 5 other changes


type: ci
user: drh
comment:
  No-op the sqlite3_memory_alarm() interface in a different way, that does
  not break legacy memory behavior.  This is a re-do of
  check-in [5d3f5df4da9f40d5].


type: ci
user: drh
comment:
  Fix the --help option on test/releasetest.tcl.  Also fix the final error
  count so that it includes a count of subtest crashes.


type: ci
user: dan
comment:
  Fix an fts5 problem that could occur if a term and the first associated rowid are on different leaf pages.


type: ci
user: dan
comment:
  Modify the fts5 leaf page format to permit faster seek operations. This is a file-format change. Any existing databases can be upgraded by running the fts5 'rebuild' command.


type: ci
user: drh
comment:
  Create separate "path" and "root" columns in the json_each() and json_tree()
  virtual tables.  "Root" is the 2nd parameter and is fixed.  
  "Path" varies as json_tree() walks the hierarchy.


type: ci
user: drh
comment:
  Fix a potential NULL pointer deref in the testing logic of pcache1.
  NB: The -DSQLITE_TEST compile-time option is needed to hit the problem.


type: ci
user: drh
comment:
  Make the sqlite3_value_subtype() and sqlite3_result_subtype() interfaces
  available to loadable extensions.


type: ci
user: drh
comment:
  Take out the goofy '$$' path syntax.  Instead, use subtypes to communicate when
  a string is JSON.  Add the json() function that validates and minifies the
  JSON and sets the appropriate subtype.


type: ci
user: drh
comment:
  Add new interfaces sqlite3_value_subtype() and sqlite3_result_subtype().
  Update the json1.c extension to take advantages of those interfaces to avoid
  the goofy '$$' path syntax and to allow nested calls to json_array() and
  json_object() that work as expected.


type: ci
user: drh
comment:
  Updates to the sqlite3_value_subtype() and sqlite3_result_subtype()
  documentation and to test cases for json1 dealing with those interfaces.


type: ci
user: mistachkin
comment:
  Enhance mutex initialization to prevent possible race conditions between sqlite3_initialize() and sqlite3_config().  Also, re-check sqlite3GlobalConfig.isInit after the mutex subsystem has been initialized.


type: ci
user: mistachkin
comment:
  Specifying an invalid mutex implementation (via SQLITE_CONFIG_MUTEX) should cause the default one to be used instead.


type: ci
user: dan
comment:
  Experimental change to use a single-pass approach for DELETE statements on non-virtual tables that do not fire triggers or require foriegn-key processing.


type: ci
user: dan
comment:
  Avoid calling sqlite3VdbeCursorMoveto() from more than one point in vdbe.c. Doing so prevents it from being inlined as part of OP_Column.


type: ci
user: dan
comment:
  Use a single-pass approach for DELETE statements on non-virtual tables that do not require trigger or foreign key processing.


type: ci
user: drh
comment:
  Use symbolic names ONEPASS_OFF, ONEPASS_SINGLE, and ONEPASS_MULTI for the
  various modes of the one-pass optimization.


type: ci
user: drh
comment:
  Add a "flush" to force incremental output to releasetest-out.txt while running
  the test/releasetest.tcl. script.


type: ci
user: dan
comment:
  Avoid passing (signed char) values directly to isspace(), isalnum() or isdigit() in json1.c. Cast the value to (unsigned char) first.


type: ci
user: dan
comment:
  Ensure that "PRAGMA integrity_check" reports an error if the free-list count header field contains a value smaller than the actual number of pages on the database free-list.


type: ci
user: drh
comment:
  Fix sqlilimits.test module so that it knows that the SELECT in a CREATE VIEW
  is not checked until the view is actually used.


type: ci
user: drh
comment:
  Fix a crash that can occur with a skip-scan on an index with
  expressions when SQLITE_ENABLE_EXPLAIN_COMMENTS is defined.


type: ci
user: drh
comment:
  Do not allow rowid in a UNIQUE or PRIMARY KEY constraint.  (This restores the
  same behavior exhibited by all prior releases.)


type: g
user: drh
comment:
  Edit [c4b68eff7f8f5402|c4b68eff7f]: Move to branch [/timeline?r=json-fuzz&nd&dp=c4b68eff7f8f5402&unhide | json-fuzz].


type: ci
user: mistachkin
comment:
  For MSVC, have the 'sqlite3.c' target depend on 'sqlite3ext.h' as well as other targets may depend on this behavior (e.g. extensions).


type: ci
user: drh
comment:
  Fix json_set() so that it can overwrite a value that was previously overwritten
  during the same call.


type: ci
user: drh
comment:
  Futher simplifications to json1.c.  Also an obscure bug-fix in the initial
  output of json_tree() when using a path to an object contained within an array.


type: ci
user: drh
comment:
  Take care that the number of reserved bits per page is consistent between 
  the source and destination databases when doing the back-copy on a VACUUM.


type: ci
user: drh
comment:
  Correctly handle the case of a WHERE-clause term with indexed expressions on
  both sides of the == sign.


type: ci
user: drh
comment:
  Strengthen the implementations of xShmMemoryBarrier on both the unix and
  windows VFSes, so that they likely work even if SQLITE_THREADSAFE=0 is used.


type: ci
user: drh
comment:
  Enhance the query planner so that it is able to use indexed expressions
  to help fulfill an ORDER BY clause.


type: ci
user: drh
comment:
  Fix a problem computing affinities for indexes during skip-scan code 
  generation when SQLITE_ENABLE_STAT4 is on.  Bug introduced by
  check-in [1ee089a72d789002].


type: ci
user: drh
comment:
  Report an error if the number of named columns in a VIEW is different
  from the number of result columns in the SELECT statement that implements
  the VIEW.


type: ci
user: drh
comment:
  Disable the SQLITE_TESTCTRL_BENIGN_MALLOC_HOOKS mechanism and replace it 
  with SQLITE_TESTCTRL_BENIGN_MALLOC_CTRL, which gives better control.


type: ci
user: drh
comment:
  Fix threads.c so that setting sqlite3FaultSim(200) using the
  SQLITE_TESTCTRL_FAULT_INSTALL callback causes both unix and windows worker
  threads to be sequential and deterministic, so that they can be tested
  reliably.


type: ci
user: drh
comment:
  Enable adding JSON1 by appending the json1.c source file to the amalgamation
  and compiling with -DSQLITE_ENABLE_JSON1


type: ci
user: drh
comment:
  Fix a database corruption bug caused by the ONEPASS optimization added
  in check-in [8b93cc5937000535].  Bug detected (prior to release) by
  [https://www.sqlite.org/sqllogictest|sqllogictest].  Test cases to follow.


type: ci
user: dan
comment:
  Changes to allow DELETE operations on virtual tables to use the onepass strategy under some circumstances.


type: ci
user: drh
comment:
  Extra information provided by .wheretrace on input flags to the query planner
  and on the result of sqlite3WhereOkOnePass().


type: ci
user: dan
comment:
  Add tests for fts3 and onepass update/delete operations. Also fix a problem with onepass updates that do not affect any rows.


type: ci
user: drh
comment:
  Change an always-true condition in the virtual table transaction
  interface into an assert().

ecomment:
  This condition was not always true after all.....

  Change an always-true condition in the virtual table transaction
  interface into an assert().


type: ci
user: dan
comment:
  Change the name of the new sqlite3_index_info.flags field to "idxFlags". Add documentation for the same to sqlite.h.in.


type: ci
user: dan
comment:
  Ensure that the xSavepoint() virtual table method is correctly invoked if there are already open savepoints (or statement transactions) the first time a virtual table is written within a transaction.


type: ci
user: drh
comment:
  Create the sqlite3IsToplevel(Parse*) interface to check to see if a top-level
  VDBE is being coded (versus a trigger) and use that interface.


type: ci
user: dan
comment:
  Add the sqlite3_index_info.idxFlags field, allowing xBestIndex() implementations to specify to SQLite that a strategy may visit at most one row. Add support for this to fts3/4. Omit the statement journal from virtual table UPDATE and DELETE operations that are guaranteed not to affect more than one row.


type: ci
user: drh
comment:
  Use symbolic names XN_ROWID and XN_EXPR in place of the (-1) and (-2)
  magic numbers in Index.aiColumn[].  Add asserts to help verify that
  Index.aiColumn[] is always used correctly.  Fix one place in FK processing
  where Index.aiColumn[] was not being used correctly.


type: g
user: drh
comment:
  Edit [3e15dea55c31c92b|3e15dea55c]: Move to branch [/timeline?r=mistake&nd&dp=3e15dea55c31c92b&unhide | mistake]. Edit check-in comment.


type: t
user: drh
brief: New ticket [1b266395d6bc1041|1b266395d6].
comment:
  New ticket [1b266395d6bc1041|1b266395d6] <i>INSERT OR REPLACE with a foreign key constraint leads to assertion fault</i>.


type: t
user: drh
brief: Ticket [1b266395d6bc1041|1b266395d6]: 6 changes
comment:
  Ticket [1b266395d6bc1041|1b266395d6] <i>INSERT OR REPLACE with a foreign key constraint leads to assertion fault</i> status still Open with 6 other changes


type: t
user: drh
brief: Fixed ticket [1b266395d6bc1041|1b266395d6].
comment:
  Fixed ticket [1b266395d6bc1041|1b266395d6]: <i>INSERT OR REPLACE with a foreign key constraint leads to assertion fault</i> plus 3 other changes


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Changes to the sesssions module ensure that tables appear within changesets and patchsets in the same order that they were attached to the sessions object.


type: ci
user: dan
comment:
  Update fts5 to avoid using a statement journal for UPDATE and DELETE operations that affect at most a single row.


type: ci
user: dan
comment:
  Add tests for the rtree module to verify that attempts to insert non-integer primary key values or non-numeric dimensions into an rtree table are handled correctly.


type: ci
user: dan
comment:
  Update fts5 to support the table function syntax. "... FROM fts5_tbl WHERE fts5_tbl MATCH ?1 AND rank MATCH ?1" can now be written "FROM fts5_tbl(?1, ?2)".


type: ci
user: dan
comment:
  Update fts3 so that expressions to the left and right of a NOT operator are balanced. This prevents relatively small expressions (a dozen terms or so) that are children of NOT operators from triggering the "expression tree is too large" error.


type: ci
user: drh
comment:
  Fix the LIMIT and OFFSET handling for UNION ALL queries that contain a 
  subquery with ORDER BY on the right-hand side.  Fix for ticket
  [b65cb2c8d91f668584].


type: t
user: drh
brief: Fixed ticket [b65cb2c8d91f6685|b65cb2c8d9].
comment:
  Fixed ticket [b65cb2c8d91f6685|b65cb2c8d9]: <i>Incorrect LIMIT on a UNION ALL query</i> plus 5 other changes


type: ci
user: mistachkin
comment:
  When running tests on Tcl 8.6 under Windows, skip tests 'uri-1.12.*' due to a change in Tcl behavior related to NTFS ADS (alternate data streams).


type: ci
user: drh
comment:
  Convert the tool/tostr.awk script into tool/tostr.tcl.  Remove two obsolete
  Makefiles.  Purge NAWK from the configure script and from unix makefiles.
  There are still two uses of NAWK in Makefile.msc.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge the version 3.9.0 changes and the incorporation of JSON1 and FTS5
  into the amalgamation from trunk.


type: ci
user: drh
comment:
  Add the JSON1 and FTS5 extensions to the amalgamation.  Add new options
  to ./configure: --enable-json1 and --enable-fts5.


type: ci
user: drh
comment:
  Add configure script options --enable-fts5 and --enable-json1.  Automatically
  search for -lm when using --enable-fts5.


type: ci
user: drh
comment:
  Fix the JSON1 extension so that it renders integers outside the range
  of -9223372036854775808 to +9223372036854775807 as floating-point numbers.


type: ci
user: drh
comment:
  The previous fix to JSON1 was not complete.  A few more tweaks are needed
  for correct handling of all oversized integers.


type: ci
user: drh
comment:
  Add some #ifdef-ery to json1.c to avoid a duplicate typedef when used in the
  amalgamation, since some compilers become upset over duplicate typedefs.


type: ci
user: mistachkin
comment:
  Fix typo in the MSVC makefile.  Replace remaining uses of AWK in the MSVC makefile with a Tcl script.


type: ci
user: drh
comment:
  Compiler warning fixes:  Rename some local variables from "j1" to avoid a
  name collision with the j1() bessel function in the math library.  Omit a
  dummy initializer that gcc 4.6.3 does not like.


type: ci
user: mistachkin
comment:
  The TclKit batch tool should not download the TclKit SDK when the TCLKIT_NOSDK environment variable is set.


type: ci
user: drh
comment:
  Improved substitution logic in the query flattener.  Saves code space, and
  (more importantly) works correctly with table-valued functions.


type: ci
user: drh
comment:
  Detect and report circularly defined views even if the views have the
  columns defined in the CREATE VIEW statement.


type: ci
user: drh
comment:
  Update makefiles to remove all uses of "awk" - to make building SQLite easier
  on Windows systems.  The only requirements now are tclsh, a C compiler, and
  common file utilities.


type: ci
user: drh
comment:
  Fix fuzzcheck.c so that it assumes that JSON1 is already compiled in and
  does not need to be initialized further.


type: g
user: drh
comment:
  Edit [8463f7e7908d0a96|8463f7e790]: Move to branch [/timeline?r=dll-build-fix&nd&dp=8463f7e7908d0a96&unhide | dll-build-fix].


type: ci
user: mistachkin
comment:
  In the MSVC makefile, add LIBTCLPATH variable to make it easier to run tests when the Tcl library is not already present in the PATH.


type: ci
user: drh
comment:
  Change sqlite3StatusSet() into sqlite3StatusHighwater() so that only the
  highwater mark is recorded.  This gives a small size reduction and
  performance improvement.


type: ci
user: drh
comment:
  Performance optimizations to the sqlite3MallocSize() by requiring the
  argument to always be non-NULL.


type: ci
user: drh
comment:
  Enhance the use of the column cache for UPDATE statements, making them
  more efficient for the case where a column is modified to be an expression
  of other unmodified columns.


type: ci
user: drh
comment:
  Whenever two or more OP_Column opcodes on the same cursor occur in succession,
  try to reordering them so that the one that extracts the right-most column is
  first, so that it will warm up the row cache for all those that follow.  This
  gives a small performance boost.


type: ci
user: drh
comment:
  Change the code generator for UPDATE to generate code in an order that might
  run more efficiently in many cases.


type: ci
user: drh
comment:
  OP_Column optimization:  Do not check for an oversize row header unless the
  row header size is larger than the content available on the local page.


type: ci
user: drh
comment:
  Use a lookup table to compute the content length for serial types less
  than 128, for a 1.2% performance improvement.


type: ci
user: drh
comment:
  Have the sqlite3VdbeSerialType() function go ahead and compute the serial
  length as well, since it is always needed.  This avoids unnecessary calls
  to sqlite3VdbeSerialTypeLen().


type: t
user: drh
brief: New ticket [57eec374ae1d0a1d|57eec374ae].
comment:
  New ticket [57eec374ae1d0a1d|57eec374ae] <i>JSON1 accepts form-feed (char(12)) as a white-space character</i>.


type: ci
user: drh
comment:
  Fix the JSON1 extension so that it does not depend on isdigit() and
  strtod() from the standard library when compiled into SQLite as part of the
  amalgamation.


type: t
user: drh
brief: Fixed ticket [57eec374ae1d0a1d|57eec374ae].
comment:
  Fixed ticket [57eec374ae1d0a1d|57eec374ae]: <i>JSON1 accepts form-feed (char(12)) as a white-space character</i> plus 5 other changes


type: ci
user: drh
comment:
  Fix the fuzzershell utility program so that it compiles with the
  amalgamation that includes JSON1 support.


type: ci
user: drh
comment:
  Merge the 3.9.1 changes into trunk.  (This only updates the version number
  and configure script since all the other changes in 3.9.1 were cherrypicked
  from trunk.)


type: ci
user: drh
ecomment:
  Cherrypick the json form-feed fix, and other #ifdef and build script changes
  to address minor issues that came to light after the 3.9.0 release.  Update
  the version number to 3.9.1.  No logic changes except for the form-feed
  bug-fix in json1 (ticket [57eec374ae1d0a1d4a]).

comment:
  Cherrypick the json form-feed fix, and other #ifdef and build script changes
  to address minor issues that came to light after the 3.9.0 release.  Update
  the version number to 3.9.0.  No logic changes except for the form-feed
  bug-fix in json1 (ticket [57eec374ae1d0a1d4a]).


type: g
user: drh
comment:
  Edit [746fcd2fd412ddc2|746fcd2fd4]: Move to branch [/timeline?r=branch-3.9&nd&dp=746fcd2fd412ddc2&unhide | branch-3.9].


type: ci
user: dan
comment:
  Handle equality and range constraints on the "term" column of fts5vocab tables. Also have the same module report columns using names instead of indexes.


type: ci
user: dan
comment:
  Fix a problem in the fts5txt2db.tcl script causing it to fail when creating tables with more than 8 columns.


type: ci
user: drh
ecomment:
  The optimization on this branch is logically incorrect.  It changes the return value of "SELECT 0 OR 'xyzzy'" FROM 0 to 'xyzzy'.

comment:
  Remove an "exit" call in the where2.test script that accidentally left in the
  code following a debugging session.


type: ci
user: dan
comment:
  Add extra debugging function to test_rbu.c. Enhance the documentation for sqlite3rbu_db() to define the validity of the returned database handles.


type: ci
user: dan
comment:
  Modifications to pass a flag to internal routine sqlite3BtreeCursor() when a cursor that is used solely for deleting b-tree entries, or for obtaining the components of keys to delete from other b-trees, is opened.


type: ci
user: dan
comment:
  Remove an unused #define from whereInt.h. Add comments describing the new sqlite3BtreeCursor() flags.


type: ci
user: dan
comment:
  Test that the binary record "0x01 0x00" is interpreted by OP_Column as a vector of NULL (or default) values.


type: t
user: dan
brief: New ticket [8a2adec166701e42|8a2adec166].
comment:
  New ticket [8a2adec166701e42|8a2adec166] <i>Problem creating automatic-indexes on sub-queries in the FROM clause of correlated scalar sub-queries</i>.


type: ci
user: dan
comment:
  When creating an automatic-index on a sub-query, add a unique integer to the end of each index key to ensure the entire key is unique. Fix for [8a2adec1].


type: t
user: dan
brief: Closed ticket [8a2adec166701e42|8a2adec166].
comment:
  Closed ticket [8a2adec166701e42|8a2adec166]: <i>Problem creating automatic-indexes on sub-queries in the FROM clause of correlated scalar sub-queries</i> plus 6 other changes


type: t
user: dan
brief: Ticket [8a2adec166701e42|8a2adec166]: 3 changes
comment:
  Ticket [8a2adec166701e42|8a2adec166] <i>Problem creating automatic-indexes on sub-queries in the FROM clause of correlated scalar sub-queries</i> status still Closed with 3 other changes


type: ci
user: drh
comment:
  When compiling with SQLITE_HAS_CODEC, honor the hexkey= query parameter on
  URI pathnames in sqlite3_open_v2().


type: ci
user: drh
comment:
  Provide the BTREE_FORDELETE flag to sqlite3BtreeCursor() if the cursor will
  be used only for deletions and seeking.


type: ci
user: drh
comment:
  Split out sqlite3BtreeCursorHintFlags() from sqlite3BtreeCursorHint() 
  the interface for improved performance.


type: ci
user: drh
comment:
  Provide hints to the storage engine using the sqlite3BtreeCursorHint()
  interface when compiling with SQLITE_ENABLE_CURSOR_HINTS.


type: ci
user: dan
comment:
  Avoid automatically rolling back the transaction if an SQLITE_IOERR or SQLITE_FULL error occurs within sqlite3_db_cacheflush().


type: ci
user: drh
comment:
  Make the internal sqlite3PExpr() interface responsive to the 
  TKFLG_DONTFOLD flag on the operator parameter.


type: ci
user: drh
comment:
  Fix uses of ctype functions (ex: isspace()) on signed characters in test
  programs and in some obscure extensions.  No changes to the core.


type: ci
user: dan
comment:
  Do not attempt to flush the pages of an in-memory database to disk if sqlite3_db_cacheflush() is called.


type: ci
user: drh
comment:
  Increase the version number to 3.10.0, due to the addition of the
  sqlite3_db_cacheflush() interface.


type: ci
user: dan
comment:
  Merge latest trunk changes, including the follow-symlinks functionality and 3.9.2 bugfixes, into this branch.


type: ci
user: dan
comment:
  On unix, if a file is opened via a symlink, create, read and write journal and wal files based on the name of the actual db file, not the symlink.


type: ci
user: dan
comment:
  Add mutex operations to test code in test3.c to avoid triggering assert() failures in certain configurations.


type: ci
user: drh
comment:
  If a table-constraint PRIMARY KEY lists a single column in single-quotes and
  that column has type INTEGER, then make that column an integer primary key,
  for historical compatibility.  Fix for ticket [ac661962a2aeab3c331].


type: t
user: drh
brief: Fixed ticket [ac661962a2aeab3c|ac661962a2].
comment:
  Fixed ticket [ac661962a2aeab3c|ac661962a2]: <i>Differing interpretation of historical schemas</i> plus 5 other changes


type: g
user: drh
comment:
  Edit [c33a275bf1a0ea10|c33a275bf1]: Edit check-in comment. Cancel background color. Cancel tag "release". Cancel tag "version-3.9.2".


type: ci
user: dan
comment:
  Add the "--jobs N" option to the releasetest.tcl script to allow tests to be run in parallel by N processes. N defaults to 1.


type: ci
user: dan
comment:
  Fix releasetest.tcl so that it does not choke if a test fails so badly that there is no log file. Add the --jobs switch to the usage message.


type: ci
user: drh
comment:
  Update the releasetest.tcl script so that it can run multiple tests in 
  parallel in separate processes.


type: g
user: drh
comment:
  Edit [4ae96d6dde7fc0a9|4ae96d6dde]: Move to branch [/timeline?r=mp-releasetest&nd&dp=4ae96d6dde7fc0a9&unhide | mp-releasetest].


type: ci
user: drh
comment:
  In the releasetest.tcl script, show the --jobs object on the command-line
  summary.  Report the time of each "starting:" event.  Do not show errors
  on a --dryrun.


type: ci
user: drh
comment:
  In the releasetest.tcl script, add the --progress option to cause the start
  of each subprocess to be shown.  Otherwise, revert to the old-style display.
  Improvements to the handling of the --buildonly option.


type: ci
user: drh
comment:
  In releasetest.tcl, run larger test first if --jobs is greater than 1.  Run
  smaller tests cases first if --jobs is 1.


type: ci
user: drh
comment:
  Rename the sqlite3PagerAcquire() function to sqlite3PagerGet().  The former
  macro called sqlite3PagerGet() has been removed.


type: ci
user: drh
comment:
  Remove a #pragma used to work around an issues with MSVC 2012 that has been
  overcome but subsequent changes.


type: ci
user: dan
comment:
  Add the 'hashsize' configuration option to fts5, for configuring the amount of memory allocated to the in-memory hash table while writing.


type: ci
user: drh
comment:
  The top of an index equality loop normally starts with OP_SeekGE and OP_IdxGT.
  This check-in adds a flag to OP_SeekGE such that it fails immediately if
  the key is not equal, then jumps over the OP_IdxGT, saving a call to the key
  comparison functions.  Consider this check-in a proof-of-concept.  It needs 
  improvement before going on trunk.  Some tests fail, but only because they
  new use fewer key comparisons than expected (which is a good thing!).


type: ci
user: drh
comment:
  Improvements and simplifications to the equality seek logic.  Tests are
  adjusted so that they all pass now.


type: ci
user: drh
comment:
  Increase the precision of integer vs. floating-point comparisons.
  Candidate fix for ticket [38a97a87a6e4e8].


type: t
user: drh
brief: New ticket [38a97a87a6e4e83e|38a97a87a6].
comment:
  New ticket [38a97a87a6e4e83e|38a97a87a6] <i>Inaccurate int/float comparison results in corrupt index</i>.


type: ci
user: drh
comment:
  A different approach to handling integer/float comparisons.  This seems to
  work better on systems where "long double" and "double" are the same type,
  but still needs refinement and testing.


type: ci
user: dan
comment:
  Have fts5 load its configuration from the xConnect() method is invoked. This ensures that the very first query run uses the correct value of the 'rank' option.


type: ci
user: drh
comment:
  A proposed patch for working around a bug in the optimizer for the HP compiler
  found on HP/UX Itanium.


type: ci
user: drh
comment:
  Enforce consistent comparison results between integer and floating point
  values.  Fix for ticket [38a97a87a6e4e83].


type: ci
user: drh
comment:
  Test changes:  Omit numindex1.test from valgrind, since valgrind uses a
  none-standard floating-point processor which gives inconsistent answers.
  Run one releasetest.tcl module using LONGDOUBLE_TYPE=double.


type: ci
user: drh
comment:
  The OPFLAG_SEEKEQ optimization is only applicable to equality comparisons
  against an index, not against a rowid table.


type: ci
user: drh
comment:
  Enhance TreeView to show WITH clauses.  Add an assert to detect the
  infinite loop behavior when certain kinds of errors occur on a nested
  WITH clause.


type: ci
user: dan
comment:
  Fix a bug in CTE handling discovered by LibFuzzer that can cause an infinite loop in the query planner.


type: ci
user: drh
comment:
  Size reduction and performance improvement in the stack-popping logic of
  the Lemon-generated parser.


type: ci
user: drh
comment:
  Avoid recursion in the yy_find_shift_action() routine of the Lemon-generated
  parser, so that routine can be inlined, for a size reduction and performance
  increase.


type: ci
user: drh
comment:
  Change the parser to use the standard "lempar.c" template over in the tool/
  folder rather than the customized "lempar.c" found in src/.


type: g
user: drh
comment:
  Edit [0a72991f4e54548f|0a72991f4e]: Move to branch [/timeline?r=parser-enhancements&nd&dp=0a72991f4e54548f&unhide | parser-enhancements].


type: ci
user: drh
comment:
  Change all parsers to use the standard "lempar.c" template in the tool/
  folder and remove the customized lempar.c from src/, plus other compiler
  performance and space enhancements.


type: ci
user: drh
comment:
  Improvements to the SQLITE_CONFIG_PAGECACHE documentation.  Enhance the
  command-line shell to be able to take advantage of the full range of
  SQLITE_CONFIG_PAGECACHE capabilities, such as setting pMem==NULL and N<0.


type: ci
user: dan
comment:
  Add a hack to debug out a description of the WHERE clause of a SELECT (or other) statement. Use this in script tool/schemalint.tcl to automatically recommend indexes that might speed up specific queries.


type: ci
user: drh
comment:
  Enhance the "PRAGMA cache_spill" statement to accept an integer argument which
  is the threshold at which spilling will begin.


type: ci
user: drh
comment:
  Add a space before each line of error output in the test scripts so that all
  errors can be conveniently located in logs by doing "grep '^ '".


type: ci
user: dan
comment:
  Add support for the SQLITE_SQLLOG_CONDITIONAL environment variable to the logging code in test_sqllog.c. When defined, logging is only performed if the "<database>-sqllog" file is present in the file system when the main database is opened.


type: ci
user: drh
comment:
  Update the SQLLOG logic so that to builds in accordance with the instructions
  in the header comment.


type: ci
user: drh
ecomment:
  Change the error messages so that they can be grepped using '^!' instead
  of '^ '.  There are far too many extraneous outputs that begin with a space.

comment:
  Change the error messages so that they begin can be grepped using '^!' instead
  of '^ '.  There are far too many extraneous outputs that being with a space.


type: ci
user: drh
comment:
  Fix the releasetest-out.txt log output from releasetest.tcl so that it works
  when the --jobs option is used.


type: ci
user: dan
comment:
  Add the "--enable-static-shell" option to the amalgamation autoconf script. If set (the default) the compiled shell tool is statically linked against sqlite3.o. Otherwise, it is linked against libsqlite3.so.


type: t
user: drh
brief: Fixed ticket [38a97a87a6e4e83e|38a97a87a6].
comment:
  Fixed ticket [38a97a87a6e4e83e|38a97a87a6]: <i>Inaccurate int/float comparison results in corrupt index</i> plus 5 other changes


type: ci
user: drh
comment:
  Code simplification in sqlite3ColumnsFromExprList().  Update the %z format
  code so that it works with buffers obtained from sqlite3DbMalloc().  Add a
  testcase for the slow column name uniquifier.


type: ci
user: dan
comment:
  In the shell tool, avoid testing if (sqlite3_vfs.xGetCurrentInt64) is NULL for a version 1 VFS. This field is only defined for version 2 and greater.


type: ci
user: dan
comment:
  Add testfixture command "vfs_current_time_int64". Returns the value returned by the xCurrentTimeInt64 method of the default VFS.


type: ci
user: drh
comment:
  The virtual table compiles but does not work and is missing many features.
  This is an incremental check-in.


type: ci
user: drh
comment:
  Import the LSM code from SQLite4 for use in an experimental virtual table.
  NB: This is a speculative experiment and could easily result in a dead-end
  branch.


type: ci
user: dan
comment:
  When using mmap mode on unix, use a read-only mapping by default. Write to the database file using write(). Unless SQLITE_MMAP_READWRITE is defined, in which case use a read/write mapping and write into the file using memcpy().


type: ci
user: mistachkin
comment:
  When using mmap mode on Win32, use a read-only mapping by default. Write to the database file using WriteFile(). Unless SQLITE_MMAP_READWRITE is defined, in which case use a read/write mapping and write into the file using memcpy().


type: ci
user: drh
ecomment:
  If a table column name begins with "__hidden__" then do not include that
  column in "*" expansions in SELECT statements, nor fill in that column in
  an INSERT INTO that omits the column list.  <i>This branch is a 
  proof-of-concept only and is not intended to ever be merged into trunk.</i>

comment:
  If a table column name begins with "__hidden__" then do not include that
  column in "*" expansions in SELECT statements, nor fill in that column in
  an INSERT INTO that omits the column list.  <b>This branch is a 
  proof-of-concept only and is not intended to ever be merged into trunk.</b>


type: ci
user: drh
comment:
  Only support the magic "__hidden__" column name prefix interpretation when
  compiled with SQLITE_ENABLE_HIDDEN_COLUMNS.


type: ci
user: drh
comment:
  If compiled with SQLITE_ENABLE_HIDDEN_COLUMNS, then columns in ordinary
  tables and views that have names beginning with "__hidden__" are omitted from
  the "*" expansion in SELECT statements and from the automatic list of columns
  following the table name in an INSERT INTO statement.


type: ci
user: drh
comment:
  Fix a 10-year-old misguided attempt at parser performance improvement that
  actually made parsing slightly slower.


type: ci
user: drh
comment:
  Create the new TK_ASTERISK token to represent the "*" in "SELECT *".  Formerly
  that operator was TK_ALL, which was also used for UNION ALL.  Less confusion if
  they operator symbols are distinct.


type: ci
user: dan
comment:
  Fix problems with INSERT INTO ... SELECT ... statements that write to tables with __hidden__ columns.


type: ci
user: drh
comment:
  When manifesting a view as part of an DELETE or UPDATE, be sure to include
  the hidden columns in the manifestation.


type: ci
user: drh
comment:
  Add a new OP_BColumn opcode where the cursor is guaranteed to be of type
  CURTYPE_BTREE and is thus able to skip some checks and run slightly faster.


type: ci
user: drh
comment:
  In the OP_Column opcode, only test the btree payload size for exceeding the
  string length limit if the payload does not fit on a single page.


type: ci
user: drh
ecomment:
  Do not allow table-valued function syntax on ordinary tables and views as those
  objects are not table-valued functions.
  Fix for ticket [73eba7ba5c7c0fc].
  Problem found by [http://llvm.org/docs/LibFuzzer.html|libFuzzer].

comment:
  Do not allow table-valued function syntax on ordinary tables and views as those
  objects are not table-valued functions.
  Fix for ticket [73eba7ba5c7c0fc].


type: t
user: drh
brief: New ticket [73eba7ba5c7c0fc6|73eba7ba5c].
comment:
  New ticket [73eba7ba5c7c0fc6|73eba7ba5c] <i>Table-valued function syntax allowed for ordinary tables and views</i>.


type: t
user: drh
brief: Fixed ticket [73eba7ba5c7c0fc6|73eba7ba5c].
comment:
  Fixed ticket [73eba7ba5c7c0fc6|73eba7ba5c]: <i>Table-valued function syntax allowed for ordinary tables and views</i> plus 5 other changes


type: ci
user: dan
comment:
  Add a rule to main.mk to build the schemalint.tcl script into an executable. Similar to the way the sqlite3_analyzer executable is built.


type: ci
user: dan
comment:
  Fix an obscure memory leak found by libfuzzer that may occur under some circumstances if expanding a "*" expression causes a SELECT to return more than 32767 columns.


type: ci
user: dan
comment:
  In the CREATE INDEX statements output by schemalint.tcl, avoid declaring an explicit collation sequence that is the same as the column's default.


type: ci
user: drh
comment:
  Do not try to eliminate No-ops at the end of VDBE program as this can cause
  problems for some DISTINCT handling algorithms, and does not improve
  performance.  This also fixes an assertion fault found by libFuzzer.


type: ci
user: drh
comment:
  Remove an incorrect ALWAYS() macro.  Fix for ticket [e5c6268dd807fa8950] - 
  a problem introduced in SQLite 3.9.0 and found by libFuzzer.


type: t
user: drh
brief: New ticket [e5c6268dd807fa89|e5c6268dd8].
comment:
  New ticket [e5c6268dd807fa89|e5c6268dd8] <i>Assertion fault on DISTINCT query containing a correlated aggregate.</i>.


type: t
user: drh
brief: Fixed ticket [e5c6268dd807fa89|e5c6268dd8].
comment:
  Fixed ticket [e5c6268dd807fa89|e5c6268dd8]: <i>Assertion fault on DISTINCT query containing a correlated aggregate.</i> plus 5 other changes


type: ci
user: drh
comment:
  Remove from os_unix.c pointless logic that tries to prevent a recurrence of
  a warning message that can only occur once.


type: ci
user: dan
comment:
  Fix a problem in whereexpr.c causing a crash while processing a user-function taht accepts zero arguments.


type: ci
user: drh
comment:
  Add the sqlite3_strlike() interface, which might be useful for implementing
  LIKE operators on virtual tables.


type: ci
user: drh
comment:
  Enhancement the virtual table interface to support LIKE, GLOB, and REGEXP
  operators.  Also add the sqlite3_strlike() interface, which might be useful
  as part of the implementation of LIKE on some virtual tables.


type: ci
user: dan
comment:
  Fix the fts5 "prefix=" option to match the documentation (space separated list, multiple prefix= options supported). The undocumented comma-separated format (compatible with fts4) still works.


type: ci
user: drh
comment:
  Simplification of the error code translator in os_unix.c.  Code cleanup only.
  The logic is unchanged.


type: ci
user: dan
comment:
  Update test_fs.c to include a virtual table that reads a file-system hierarchy. Use it to further test GLOB and LIKE support for virtual tables.


type: ci
user: drh
comment:
  Fix harmless compiler warnings in test_fs.c.  Fix typos and clean up the text
  of the documentation for sqlite3_strglob() and sqlite3_strlike().


type: ci
user: dan
comment:
  Fix a problem with the userauth extension and no-authentication databases. Run the tests for this extension as part of the Debug-One module in releasetest.tcl.


type: ci
user: dan
comment:
  Add the "colUsed" field to the sqlite3_index_info structure passed to virtual table xBestIndex methods. To indicate the subset of the virtual table columns that may be required by the current scan.


type: ci
user: drh
comment:
  The first argument to SQLITE_CONFIG_PAGECACHE, SQLITE_CONFIG_SCRATCH, and
  SQLITE_CONFIG_HEAP must always be a pointer.


type: ci
user: drh
comment:
  Add the SQLITE_FCNTL_VFS_POINTER file control which obtains a pointer to
  the top-level VFS in use by a database connection.


type: ci
user: drh
comment:
  Fix the sqldiff utility program so that it works for schemas that have
  tables with zero-length column names.


type: ci
user: drh
comment:
  Fix the threadtest3 test program so that it works with VFSes that omit the
  xCurrentTime() method and supply only xCurrentTimeInt64().


type: ci
user: drh
comment:
  The EOVERFLOW errors from fstat() is not possible unless SQLite has been
  compiled with SQLITE_DISABLE_LFS.


type: ci
user: drh
comment:
  Fix a problem in xFullPathname for the unix VFS.  The problem was found by
  Kostya Serebryany using libFuzzer.


type: ci
user: drh
comment:
  Simplification to the posix_fallocate() replacement used for the
  SQLITE_FCNTL_SIZE_HINT file control in the unix VFS.


type: ci
user: drh
comment:
  Remove all traces of SQLITE_FCNTL_WAL_BLOCK from the unix VFS - that feature
  had been disabled for a long time and never actually worked.


type: ci
user: drh
comment:
  Fix a (harmless) off-by-one error in the unix VFS logic that fsync()s a
  directory after deleting a file.


type: ci
user: drh
comment:
  Change the SQLITE_NO_SYNC compile-time option to call fstat() in place of
  fsync() rather than being a total no-op.


type: ci
user: drh
comment:
  Remove more (dead) SQLITE_FCNTL_WAL_BLOCK logic from wal.c - code that was
  missed during the [e1d5320ca08933] check-in.


type: ci
user: dan
comment:
  Have the sqlite3_column_decltype() API report the declared types for the left-most SELECT statement in a compound SELECT.


type: ci
user: drh
comment:
  Remove the dependence on "exec ls -U" from the vtabH.test module, as the -U
  option to "ls" is not universally available.


type: ci
user: dan
comment:
  Improve concurrency in test_multiplex.c. Add a switch to "threadtest3" allowing it to run using the multiplexor VFS.


type: ci
user: drh
comment:
  A unix VFS change replaces fsync() with fstat() when using SQLITE_NO_SYNC, so
  set PRAGMA synchronous=OFF in the sysfault-3 test to avoid erroneously causing
  errors in xSync.


type: ci
user: drh
comment:
  Make the sqlite3_status64(), sqlite3_strlike(), and
  sqlite3_db_cacheflush() APIs available to loadable extensions.


type: ci
user: drh
comment:
  Cleaner code and additional comments on the handling of 8+3 filenames when
  trying to find the name of a database file based on its journal filename, 
  in the unix VFS.


type: ci
user: drh
comment:
  Prevent a segfault on Solaris in the test_fs.c due to differences in the
  definition of the dirent object.


type: ci
user: drh
comment:
  Fix the openDirectory() routine in the unix VFS so that it works for databases
  located in the root of the filesystem and for database files that have no
  pathname at all.


type: ci
user: drh
ecomment:
  Changes to avoid undefined behavior detected by analysis tools - never
  observed in the wild.
  <i>Later:</i> This check-in introduces a bug, fixed at [a304e34675404aee].

comment:
  Changes to avoid obscure, theoretical undefined behavior. This is preventative
  measures only - no actual problems observed on tested compilers.


type: ci
user: drh
comment:
  Add the ".changes ON|OFF" command to the sqlite3.exe command-line shell, for
  testing and verifying the sqlite3_changes() and
  sqlite3_total_changes() interfaces.


type: ci
user: drh
comment:
  Add a test case of the form "WHERE a<2 OR a<3" using PRAGMA count_changes.
  This test case was failing before the 3.9.0 release.


type: ci
user: drh
comment:
  Changes to avoid undefined behavior in memset() and memcpy() and in the
  comparisons of pointers from different allocations.  All problems are found
  by analysis tools - none have been seen in the wild.


type: ci
user: drh
comment:
  Avoid doing comparisons with pointers that might have been previously been
  passed to realloc() and/or free().


type: ci
user: dan
comment:
  Experimental optimization for DELETE statements with WHERE clauses that qualify for the OR-optimization.


type: ci
user: dan
comment:
  Update sqlite3_snapshot_open() to reduce the chances of reading a corrupt snapshot created by a checkpointer process exiting unexpectedly.


type: ci
user: drh
comment:
  Simplification of the DROP TRIGGER logic using sqlite3NestedParse() instead
  of hand-coded VDBE code.  This is a manual cherry-pick of the key change from
  check-in [c80bbf14b365d].


type: ci
user: drh
comment:
  Add the nBackfillAttempted field in formerly unused space in WalCkptInfo and
  use that field to close the race condition on opening a snapshot.


type: ci
user: dan
comment:
  Have sqlite3_snapshot_open() avoid a race condition by taking a shared CHECKPOINTER lock while checking pInfo->nBackfillAttempted.


type: ci
user: dan
comment:
  Add tests to ensure that an sqlite3_snapshot_open() client cannot be tricked into reading a corrupt snapshot even if another process fails mid-checkpoint.


type: g
user: drh
comment:
  Edit [ad3124c834b080aa|ad3124c834]: Move to branch [/timeline?r=stdint.h&nd&dp=ad3124c834b080aa&unhide | stdint.h].


type: ci
user: drh
comment:
  Fix a memory allocation bug introduced last week by check-in
  [a9e819082ba].  The bug only appears on systems where the size of a structure
  is not always a multiple of 8 - which in practice means only on 32-bit 
  windows systems.


type: ci
user: dan
comment:
  Return SQLITE_BUSY (not SQLITE_BUSY_SNAPSHOT) if sqlite3_snapshot_open() fails to obtain the shared checkpointer lock.


type: ci
user: dan
comment:
  Update the sqlite3_snapshot_get() API so that if the caller does not have an open read transaction on the named database, one is automatically opened.


type: ci
user: drh
comment:
  Mention that the snapshot interfaces are only available if SQLite is compiled
  with SQLITE_ENABLE_SNAPSHOT.


type: ci
user: drh
comment:
  Add the experimental snapshot interface.  Because it is experimental, it is
  subject to change or removal at a later date.


type: ci
user: dan
comment:
  Add further tests for the changes on this branch. Also fix a memory-leak that could follow a malloc failure.


type: ci
user: dan
comment:
  Add a missing memAboutToChange() macro to vdbe.c, the lack of which was causing an assert() to fail incorrectly. Problem found by libfuzzer.


type: ci
user: dan
comment:
  Update the mkautoconfamal.sh script to make the building of the sqlite3-autoconf-VERSION.tar.gz package more orthodox.


type: ci
user: drh
comment:
  Make the VdbeCursor object even smaller.  But the resulting library is bigger
  and slower than trunk.  This branch is a failed experiment.


type: ci
user: drh
comment:
  Fix the spellfix1_scriptcode() function to ignore whitespace and punctuation,
  and to recognize hebrew and arabic scripts.


type: ci
user: dan
comment:
  Add the "offsets=0" option to fts5, to create a smaller index without term offset information. A few things are currently broken on this branch.


type: ci
user: dan
comment:
  Updates so that fts5 API functions xInst, xPhraseFirst and xPhraseNext work with the offsets=0 option.


type: ci
user: drh
comment:
  Ensure that the Expr objects that describe indexed expressions are not modified
  by code generation.  Fix for an assert() problem found by Jon Metzman using AFL.


type: ci
user: drh
comment:
  Enhance the 'utc' modifier on date/time functions so that if the LHS is
  already known to be in UTC, the modifier becomes a no-op.  This is not an
  incompatibility because the behavior is documented as "undefined" in that
  scenario.


type: ci
user: drh
comment:
  Avoid a harmless compiler warning on systems where the byteorder cannot
  be determined at compile-time.


type: ci
user: dan
comment:
  Add the xPhraseFirstColumn() and xPhraseNextColumn() API functions to fts5. For iterating through the set of columns that contain intances of a phrase.


type: ci
user: dan
comment:
  Change the name of the offsets=0 option to "detail=column". Have the xInst, xPhraseFirst and other API functions work by parsing the original text for detail=column tables.


type: ci
user: drh
comment:
  Remove unnecessary tests from the LIKE pattern matcher.  Slightly faster and
  slightly smaller, and it should also now works with EBCDIC.


type: ci
user: drh
comment:
  Changes to the way that the default BINARY collating sequence is recorded
  result in a slightly smaller and slightly faster executable.  More work could
  be done to make this cleaner.


type: ci
user: drh
comment:
  Reduce the size of the CellInfo object from 32 to 24 bytes on 64-bit machines,
  for a small performance increase and library size reduction.


type: ci
user: dan
comment:
  Changes to run many fts5 tests with detail=none and detail=col tables as well as the default detail=full. Also fixes for the bugs uncovered by running said tests.


type: ci
user: drh
comment:
  Fix the ieee754.test module so that it works both on systems that report
  infinity as "inf" and as "Inf".


type: ci
user: drh
comment: Avoid misaligned memory allocations on Sparc in sqlite3VdbeMakeReady().
ecomment:
  Avoid misaligned memory allocations on Sparc in sqlite3VdbeMakeReady().
  This fixes a problem introduced by check-in [a9e819082b].


type: ci
user: drh
comment:
  Add the number of cores used and the name of the host computer to the
  summary at the end of releasetest.tcl.


type: ci
user: dan
comment:
  Update more fts5 tests to run in detail=none and detail=column modes as well as the default detail=full.


type: ci
user: drh
comment:
  Bug fix: the conflict resolution behavior for the implied NOT NULL constraint
  on the PRIMARY KEY of a WITHOUT ROWID table should be ABORT.


type: ci
user: drh
comment:
  Limit the length of sqlite3_mprintf() output to 2^30 bytes, even if
  SQLITE_MAX_LENGTH is set larger at compile-time.


type: ci
user: drh
comment:
  Adjust the vtabH.test module to ignore files in the root directory of Windows
  machines that contain the "$" character.


type: ci
user: drh
comment:
  A new approach to very large sqlite3_mprintf() strings:  Back out the
  [d655a665] check-in and instead make the size fields in StrAccum unsigned.
  Strings generated by sqlite3_mprintf() can now be as large as 2^31-1 bytes.


type: ci
user: dan
comment:
  Fix an OOM handling problem in fts5 detail=none and detail=col modes. Also a bug in the xInst() API when used with detail=col and column filtering expressions. Update the matchinfo() test function so that 'b' is fast with detail=col tables.


type: ci
user: drh
comment:
  Fix a corner case in the opcode-array reuse logic where the number of bytes of
  reusable space might be computed to be a negative number, due to unusual
  system alignment restrictions and rounding error.


type: ci
user: drh
comment:
  Another attempt to get reuse of excess opcode array space working correctly
  on all architectures and platforms.


type: ci
user: dan
comment:
  Fix a problem with the xPhraseFirstColumn() API in detail=col mode. Remove some unused test code from fts5ac.test.


type: ci
user: dan
comment:
  Fix an fts5 problem involving detail=none, "ORDER BY rowid DESC" and deleted items. Also add tests to verify that the documented operator precedences are correct.


type: ci
user: drh
comment:
  Disable the --limit-mem option on fuzzcheck unless the SQLITE_ENABLE_MEMSYS5 or
  SQLITE_ENABLE_MEMSYS3 compile-time options are used.


type: ci
user: dan
comment:
  Have the vdbe layer call sqlite3BtreeEnter() on all b-trees in use from within sqlite3VdbeExec() even in SQLITE_THREADSAFE=0 builds. This ensures that BtShared.db is set correctly.


type: ci
user: drh
comment:
  Add the "sqlite3 -sourceid" command in the TCL interface.  Use that command and
  others to output additional information about the test configuration at the 
  conclusion of a TCL test.


type: ci
user: drh
comment:
  Fix to cause BtShared.db to be set correctly on shared-cache
  connections in SQLITE_THREADSAFE=0 builds.  Added assert()s to verify
  the correct setting of BtShared.db.


type: ci
user: drh
comment:
  Omit performance tracking from the memsys5 memory allocator if neither
  SQLITE_DEBUG nor SQLITE_TEST are defined.


type: ci
user: drh
comment:
  Add the experimental SQLITE_FCNTL_JOURNAL_POINTER file control for obtaining
  the sqlite3_file pointer associated with a rollback journal or WAL file.


type: ci
user: drh
comment:
  Fix a problem with #ifdefs on the system calls in the unix VFS that causes
  problems when compiling with SQLITE_OMIT_WAL.


type: ci
user: dan
comment:
  If a single page is written to the wal file more than once, have each subsequent copy overwrite the original frame.


type: ci
user: dan
comment:
  If it is known that checksums will be recalculated on transaction commit, skip calculating checksums when appending frames to the wal file. When recalculating checksums, recalculate them starting with the first overwritten frame - not the first frame in the transaction.


type: ci
user: drh
comment:
  All WAL frame overwrites even if there are active savepoints.  This is safe
  because a ROLLBACK TO will cause all reverted pages to be rewritten to the
  WAL file prior to COMMIT.


type: ci
user: dan
ecomment:
  If a single page is written to the wal file more than once, instead of appending the second and subsequent copy to the wal file, overwrite the first.
  Update: See the important bug fix at [f694e60a]!

comment:
  If a single page is written to the wal file more than once, instead of appending the second and subsequent copy to the wal file, overwrite the first.


type: ci
user: drh
comment:
  Make sure all bytes of the WAL file are initialized even if the checksums are
  not being computed until commit due to WAL overwrite.


type: ci
user: drh
comment:
  Fix a harmless use of an uninitialized variable following system errors
  in the multi-threaded sorter.


type: ci
user: drh
comment:
  When the [https://www.sqlite.org/queryplanner.html#partialsort|block sorting optimization]
  is used in a scalar subquery, be sure to exit the loop as soon as the first
  valid output row is received.  Fix for ticket [cb3aa0641d9a4].


type: t
user: drh
brief: New ticket [cb3aa0641d9a4138|cb3aa0641d].
comment:
  New ticket [cb3aa0641d9a4138|cb3aa0641d] <i>Incorrect result from a subquery containing over-constrainted ORDER BY</i>.


type: t
user: drh
brief: Ticket [cb3aa0641d9a4138|cb3aa0641d]: 6 changes
comment:
  Ticket [cb3aa0641d9a4138|cb3aa0641d] <i>Incorrect result from a subquery containing over-constrainted ORDER BY</i> status still Open with 6 other changes


type: t
user: drh
brief: Fixed ticket [cb3aa0641d9a4138|cb3aa0641d].
comment:
  Fixed ticket [cb3aa0641d9a4138|cb3aa0641d]: <i>Incorrect result from a subquery containing over-constrainted ORDER BY</i> plus 3 other changes


type: ci
user: drh
comment:
  Look for pthread_mutexattr_init() instead of pthread_create() in autoconf
  scripts, to work around issues with ASAN.


type: ci
user: drh
comment:
  Case folding in the LIKE operator should look at all bits of the character,
  not just the lower 8 bits.  Fix for ticket [80369eddd5c94].  This is a
  back-out of check-in [0a99a8c4facf] with a testcase added.


type: g
user: drh
comment:
  Edit [254419c36766225c|254419c367]: Change background color to "#d0c0ff". Edit check-in comment. Timestamp 2016-01-14 00:01:23. Add tag "release".


type: ci
user: drh
comment:
  Use sqlite3VdbeAddOp4() rather than a separate sqlite3VdbeChangeP4() call, for
  a slightly smaller and faster binary.


type: ci
user: drh
comment:
  Improvements to the way sqlite3VdbeAddOpList() works, resulting in a slightly
  smaller and faster binary.


type: ci
user: drh
comment:
  During compilation, transfer the azVar[0..nzVar-1] array from Parse to Vdbe
  rather than copying it.  Smaller and faster code.


type: ci
user: drh
comment:
  Suppress the display of the P4 operand in EXPLAIN output when an opcode
  has been converted into a No-op.


type: ci
user: dan
comment:
  Fix a problem in autoconf/configure.ac causing --enable-readline to fail if libedit was not present.


type: t
user: drh
brief: Fixed ticket [80369eddd5c94d49|80369eddd5].
comment:
  Fixed ticket [80369eddd5c94d49|80369eddd5]: <i>Incorrect case folding in the LIKE operator</i> plus 5 other changes


type: ci
user: drh
comment:
  Change the automatic index mechanism so that it avoids creating transient
  indexes on columns that are known to have low cardinality.


type: g
user: drh
comment:
  Edit [17efb4209f97fb49|17efb4209f]: Change background color to "#d0c0ff". Edit check-in comment. Add tag "release".


type: ci
user: drh
comment:
  Add a new hint bit on the flags parameter of sqlite3BtreeDelete().  The new
  BTREE_IDXDELETE bit indicates that the call is to delete an index entry 
  corresponding to a table row that has already been deleted.


type: ci
user: dan
comment:
  Update the schemalint.tcl script so that the argument to a -select option may be either an SQL statement or the name of a file containing an SQL statement


type: ci
user: dan
comment:
  Have the dbstat module dequote any argument passed to the CREATE VIRTUAL TABLE statement before attempting to match it against the names of attached databases.


type: ci
user: drh
comment:
  Reenable the xCurrentTime and xGetLastError methods on the unix VFSes as long
  as SQLITE_OMIT_DEPRECATED is not defined.


type: ci
user: drh
comment:
  In the TCL interface, if a database connection object was opened with
  the -uri 1 option, then also honor URI filenames for the "backup" and
  "restore" commands.


type: ci
user: mistachkin
comment:
  Add tooling to create an MSVC Makefile capable of building the core library and shell only, using a pre-built amalgamation.


type: ci
user: dan
comment:
  Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode.


type: ci
user: drh
comment:
  Add the SQLITE_SPELLFIX_STACKALLOC_SZ compile-time option to control how much
  stack space is available for use as the Wagner matrix in editDest3 of the
  spellfix extension.


type: ci
user: drh
comment:
  Changes to spellfix to try to get it to use stack space instead of heap space
  in cases where that makes sense.


type: ci
user: drh
comment:
  Adjust the amalgamation-tarball build process so that it includes the
  MSVC makefile and the sqlite3.rc resource file.


type: ci
user: drh
comment:
  Move Makefile.min.msc into the autoconf subdir where it belongs and change
  its name to Makefile.msc.  Adjust build scripts accordingly.


type: ci
user: drh
comment:
  Small simplification to the prepare statement opcode memory reuse logic.
  Easier to read, and slightly smaller and faster.


type: ci
user: dan
comment:
  Fix issues on unix with opening database files via symlinks that are not in the current working directory. And with nested symlinks.


type: ci
user: drh
comment:
  Fix a compiler warning about doing pointer arithmetic involving a NULL pointer
  even though the result of computation is never used.


type: ci
user: dan
comment:
  Ensure that unixFullpathname() always nul-terminates its output buffer, even when returning an error.


type: ci
user: drh
ecomment:
  Take care to clear the subtype on VDBE registers when the value of the
  register is overwritten with new content.

comment:
  Take care to clear the subtype on VDBE registers when the value of the
  register is overwritten with new content.
  Fix for ticket [f45ac567eaa9f93].


type: ci
user: dan
comment:
  Further minor performance improvements and code-size reductions related to fts5 column filters on detail=col tables.


type: ci
user: drh
comment:
  Add assert() statements on the nExtraDelete variable in vdbe.c to try to verify
  that the FORDELETE and IDXDELETE flags are being generated correctly.  Those
  flags are not currently generated correctly, and so the assert()s trip on this
  check-in.


type: ci
user: dan
comment:
  Modify the order of terms in an "if" condition to avoid implicitly comparing a dangling pointer to NULL following an OOM error.


type: ci
user: drh
comment:
  Avoid unnecessary WHERE clause term tests when coding a join where one
  of the tables contains a OR constraint.


type: ci
user: drh
comment:
  Avoid unnecessary WHERE clause term tests when coding a join where one of the tables contains a OR constraint.


type: g
user: drh
comment:
  Edit [ab94603974a0ad53|ab94603974]: Move to branch [/timeline?r=OR-clause-improvement&nd&dp=ab94603974a0ad53&unhide | OR-clause-improvement].


type: ci
user: drh
comment:
  Improvements to FORDELETE processing, including bug fixes and adding
  assert() statements to help verify correct operation.


type: ci
user: drh
comment:
  Add the BTREE_FORDELETE and BTREE_AUXDELETE flags to the b-tree layer interface
  and use them.  Add assert() statement to verify that they are correct.


type: ci
user: drh
comment:
  Change the P4_INTARRAY so that always begins with a length integer.  Print
  the content of the INTARRAY on EXPLAIN output.


type: ci
user: drh
comment:
  Simplification and size reduction to the printf logic.  Remove the bFlags
  parameter from sqlite3VXPrintf() and sqlite3XPrintf().  Use sqlite3XPrintf()
  instead of sqlite3_snprintf() for rendering P4 values in EXPLAIN output.


type: ci
user: drh
comment:
  Add the SQLITE_DBCONFIG_REQUIRE_WRITE_TXN connection setting, which if enabled
  requires all write operations to be enclosed within BEGIN ... COMMIT.


type: t
user: drh
brief: Fixed ticket [f45ac567eaa9f93c|f45ac567ea].
comment:
  Fixed ticket [f45ac567eaa9f93c|f45ac567ea]: <i>Invalid JSON generated by json_group_array()</i> plus 7 other changes


type: ci
user: drh
comment:
  Only honor the register subtype value if the MEM_Subtype flag is set.
  Revised and improved fix for ticket [f45ac567eaa9f9].


type: ci
user: dan
comment:
  A different way to clear the subtype on VDBE registers when the value of the register is overwritten with new content. To fix [[f45ac567eaa9f93].


type: g
user: drh
comment:
  Edit [ec653cbcae74ecab|ec653cbcae]: Move to branch [/timeline?r=clear-subtype-fail&nd&dp=ec653cbcae74ecab&unhide | clear-subtype-fail]. Mark "Closed". Edit check-in comment.


type: ci
user: drh
comment:
  Merge the implementation of OP_IdxRowid and OP_Seek so that OP_Seek no longer
  requires the rowid register and a separate OP_IdxRowid call.  Shorter and
  faster prepared statements result.


type: ci
user: drh
comment:
  Minor simplification to the OP_AutoCommit opcode.  Fix some code comments
  related to transaction control.


type: ci
user: dan
comment:
  Fix a problem causing the OR/covering-index optimization to be disabled if compile time parameter SQLITE_MAX_ATTACHED were set to greater than 30.


type: ci
user: dan
comment:
  Update walcrash.test to ensure that, during a particular test, enough data is written to cause SQLite to sync the wal file 14 times.


type: ci
user: drh
comment:
  Number VDBE opcodes starting with 0 instead of 1, as this obviates the
  lower-bound test on "switch(opcode){...}", making the code smaller and faster.


type: ci
user: drh
comment:
  Add and use the sqlite3VdbeZeroRegister() interface for coding the common
  operation of writing zero into a register.


type: ci
user: drh
comment:
  Enhance the comment on the sqlite3_index_constraint object to bring attention 
  to the fact than iColumn field can be negative for a rowid.


type: ci
user: drh
comment:
  Merge OOM handling optimizations and PRAGMA synchronous=EXTRA as well as
  other enhancements from trunk.


type: ci
user: drh
comment:
  Enhance the internal sqlite3VdbeAddOpList() interface to automatically update
  jump destinations.  Use this feature to simplify the AUTOINCREMENT code
  generator.


type: ci
user: drh
comment:
  Add "PRAGMA synchronous=EXTRA" which syncs the directory containing the
  rollback journal when the rollback journal is deleted in order to commit a 
  transaction in DELETE mode.


type: ci
user: drh
comment:
  Fix markup errors in comments used to generate the documentation - specifically
  in the documentation on the OP_Seek opcode.


type: ci
user: drh
comment:
  In the VDBE loop, only check for OOM errors at jumps rather than after every
  opcode, for about a 0.5% performance increase.


type: ci
user: drh
comment:
  Refinements to synchronous logic: 
  (1) Use PAGER_SYNCHRONOUS_FULL rather than the corresponding magic number.
  (2) Honor SQLITE_NO_SYNC on xDelete calls with sync
  (3) Count xDelete syncs during testing
  (4) Fix #ifs on SQLITE_EXTRA_DURABLE so that directory syncs on journal
  unlink are off by default.


type: t
user: drh
brief: Fixed ticket [ad2559db380abf8e|ad2559db38].
comment:
  Fixed ticket [ad2559db380abf8e|ad2559db38]: <i>JSON does not properly escape control characters</i> plus 5 other changes


type: ci
user: drh
comment:
  Temporarily back out the 0.5% performance improvement from
  check-in [632071bac5ff32].  Need a more elaborate solution that works
  with reentrant virtual tables and SQL functions.


type: ci
user: drh
comment:
  Add the slightly faster sqlite3DbMallocRawNN(db,n) routine for the majority
  cases where db is guaranteed to be not NULL.


type: ci
user: drh
comment:
  Improvements to the way out-of-memory conditions are handled, resulting
  in an almost 1% performance gain in the commmon case where OOM errors do
  not occur.


type: ci
user: drh
comment:
  Make sure the "bak.db" database file does not actually exist before
  starting the "quota.test" testing.


type: ci
user: drh
comment:
  Add a utility program that looks for assert(), NEVER(), ALWAYS(), and
  testcase() macros that have side-effects, and reports errors when they are
  found.  Also fix a bug that this utility detected as it was being tested.


type: ci
user: drh
comment:
  Faster keywordCode() implementation by taking advantage of the fact that
  the input is always pure ASCII alphabetic and underscore and that the keyword
  table is always upper-case.


type: ci
user: drh
comment:
  Demonstrate a much faster sqlite3GetToken() routine by using a lookup table
  to map initial token characters into a character class.  This check-in does
  not work for EBCDIC.  More optimization needed.


type: ci
user: dan
comment:
  Disable the two-argument form of the fts3_tokenizer() SQL function unless the library is built with -DSQLITE_ENABLE_FTS3_TOKENIZER.


type: ci
user: drh
comment:
  Make sure every co-routines has its own set of temporary registers and does
  not share temporaries, since a co-routine might expect the content of a
  temporary register to be preserved across an OP_Yield.
  Proposed fix for ticket [d06a25c84454a].


type: t
user: drh
brief: Fixed ticket [d06a25c84454a372|d06a25c844].
comment:
  Fixed ticket [d06a25c84454a372|d06a25c844]: <i>Incorrect result from a UNION with an ORDER BY</i> plus 6 other changes


type: ci
user: drh
comment:
  Fix the top-level makefiles to always include SQLITE_ENABLE_EXPLAIN_COMMENTS
  when building the shell.  Still need to fix main.mk and the
  amalgamation tarball.


type: ci
user: drh
comment:
  Code simplification:  ALTER TABLE ADD COLUMN always upgrades the file_format
  to 4 if is not there already.  No need to upgrade to only 2 or 3 since format
  4 has now been supported for over 10 years.


type: ci
user: drh
comment:
  Add auto-explain mode to the command-line shell.  Default on.  Auto-explain
  tries to automatically detect EXPLAIN queries and format them appropriately.


type: ci
user: drh
comment:
  Enhance the command-line shell to be in auto-explain mode by default.  It is
  no longer necessary to use the ".explain" command to put the shell into a mode
  where the EXPLAIN output is formatted nicely.  That now happens automatically.


type: ci
user: drh
comment:
  Fix the makefiles in the autoconf-tarball so that they build the command-line
  shell with SQLITE_ENABLE_EXPLAIN_COMMENTS.


type: ci
user: drh
comment:
  Add the new "snapshot-tarball" target to the unix makefiles, for building
  a tarball similar to "amalgamation-tarball" but named by the SHA1 hash of
  the current check-out rather than by the version number.


type: ci
user: drh
comment:
  Update the snapshot tarball name to use the date (ISO8601 format) instead
  of the SHA1 hash in its name.


type: ci
user: drh
comment:
  When generating the snapshot-tarball, truncate the date/time in the name to 
  12 significant digits (YYYYMMDDhhmm) omitting the seconds and fractional
  seconds.


type: ci
user: drh
comment:
  Omit unnecessary CHECK constraints in UPDATE statements, when none of the
  columns referenced in the CHECK constraint are modified.


type: ci
user: drh
comment:
  Fix to previous check-in: Make sure CHECK constraints involving the ROWID
  are not ignored when the ROWID changes.


type: ci
user: drh
comment:
  Add testcase() macros to the CHECK constraint avoidance logic.  Avoid creating
  an unused VDBE label during CHECK constraint code generation.


type: ci
user: mistachkin
comment:
  In the MSVC makefile, include the working directory so that it can pickup build generated files (e.g. sqlite3.h, etc).


type: ci
user: dan
comment:
  Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code.


type: ci
user: drh
comment:
  Add JSON1 and FTS5 to the set of extensions subject to close compiler warning
  analysis.  Fix some warnings in each.   More (harmless) warnings still exist
  in FTS5.


type: ci
user: mistachkin
comment:
  Prevent 'expanded command line X too long' errors during the batch 'for' loops used for copying files.


type: ci
user: drh
comment:
  Remove the unexplained extra 3.0 cost factor for the B-tree sorting that is
  done to implement DISTINCT in the absence of an index to help.


type: ci
user: drh
comment:
  Enhance mptester so that it takes great care to delete its test database
  prior to start-up, including checking the return code from unlink() and
  retrying after a delay if unlink() fails.  Hopefully this will clear 
  intermittant startup problems on Windows.


type: ci
bgcolor: #b1d3a8
user: dan
comment:
  Fix a problem in sessionfault.test causing it to segfault following a test failure. This commit does not fix the actual test failure - just the subsequent segfault.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Harden sqlite3session_delete() against trying to delete a session that
  is not currently on the session list.


type: ci
user: drh
comment:
  Provide Sqlite3_SafeInit() and Sqlite3_SafeUnload() entry points on the TCL
  interface, but have the always return TCL_ERROR, because the non-standard 
  TCL builds on Macs require this.


type: ci
user: drh
comment:
  Add the SQLITE_NOMEM_BKPT macro to enhance the ability to debug OOM errors.
  Only effective with compiling with SQLITE_DEBUG.


type: ci
user: drh
comment:
  Make the application-defined SQL function logic more compact and faster,
  especially faster for the case where an application defines thousands of
  new SQL functions.


type: ci
user: drh
comment:
  Improvements to the application-defined function mechanism so that it is
  more compact and runs faster, especially when the application defines
  thousands of new SQL functions.


type: ci
user: drh
comment:
  In the tool/warning.sh script, disable Android-specific compiler warning tests
  when not running on Linux.


type: ci
user: drh
comment:
  On windows systems when rand_s() is available, use it to obtain additional
  seed material in winRandomness().


type: ci
user: drh
comment:
  Improve the Lemon parser template (lempar.c) so that it avoids unnecessary
  work when the grammer defines YYNOERRORRECOVERY (as SQLite does).  Slightly
  smaller and faster code results.


type: ci
user: dan
comment:
  Schemalint changes: Avoid creating candidate indexes if a compatible index exists. Do not quote identifiers that do not require it.


type: ci
user: drh
comment:
  In Lemon, add the ability for the left-most RHS label to be the same as the
  LHS label, causing the LHS values to be written directly into the stack.


type: ci
user: drh
comment:
  Enhance Lemon so that if reduce code contains a comment of the form
  "/*A-overwrites-X*/" then a LHS label A is allowed to overwrite the
  RHS label X.


type: ci
user: drh
comment:
  More agressive use of /*A-overwrites-X*/ in the parser.  Fix an off-by-one
  error in parser stack overflow detection.


type: ci
user: drh
comment:
  Enhance Lemon to generate more compact and efficient code for yy_reduce().
  Update the main SQL grammar to take advantage of the new capabilities.


type: ci
user: dan
comment:
  Remove the "#ifdef SQLITE_TEST" condition from fts5_test_mi.c. Update fts5_main.c so that if SQLITE_FTS5_TEST_MI is defined when it is built the fts5_test_mi.c entry point is automatically called. This makes it easier to build versions of fts5 that include the matchinfo() demo by default.


type: ci
user: drh
comment:
  Fix up all VDBE opcodes so that they cause an immediate exit on any kind of
  error, and thereby eliminate the need to test "rc" at the top of the loop.
  Resulting code is a little smaller and faster.


type: ci
user: drh
comment:
  When VDBE opcodes fail, they cause an immediate exit from the loop, rather
  than depending on the rc==SQLITE_OK test at the top of the loop.  This is
  faster and smaller.


type: ci
user: drh
ecomment:
  Avoid a potential buffer overrun if an SQL statement being parsed ends
  with an illegal "!" token.  (This problem was detected by fuzzcheck
  running under valgrind. The problem was introduced by check-in [9570b6b43df3],
  two days ago and has not appeared in a release.)

comment:
  Avoid a potential buffer overrun if an SQL statement being parsed ends
  with an illegal "!" token.  (This problem was detected by fuzzcheck
  running under valgrind. The problem was introduced by check-in [9570b6b43df3].)


type: ci
user: dan
comment:
  Add an "#ifndef SQLITE_AMALGAMATION" guard around a typedef in fts5_test_mi.c to avoid redefining "u32" if it is compiled as part of the amalgamation. Cherrypick of [f9c49ba1].


type: ci
user: dan
comment:
  Remove SQLITE_TEST macros from fts5_test_mi.c. Add the SQLITE_FTS5_ENABLE_TEST_MI compile time symbol - to make it easier to create builds that include the fts5 matchinfo demo by default.


type: ci
user: dan
comment:
  Add an "#ifndef SQLITE_AMALGAMATION" guard around a typedef in fts5_test_mi.c to avoid redefining "u32" if it is compiled as part of the amalgamation.


type: ci
user: dan
comment:
  Fix an fts5 problem causing 'optimize' to corrupt the fts index under some circumstances. Cherrypick of [251d6473f7].


type: ci
user: dan
comment:
  Use a separate list of aux-data structures for each trigger program at the VDBE level. Fix for [dc9b1c91].


type: t
user: dan
brief: Ticket [dc9b1c91db15ae9d|dc9b1c91db]: 6 changes
comment:
  Ticket [dc9b1c91db15ae9d|dc9b1c91db] <i>One trigger may use auxdata belonging to another.</i> status still Open with 6 other changes


type: t
user: dan
brief: Closed ticket [dc9b1c91db15ae9d|dc9b1c91db].
comment:
  Closed ticket [dc9b1c91db15ae9d|dc9b1c91db]: <i>One trigger may use auxdata belonging to another.</i> plus 2 other changes


type: t
user: dan
brief: Ticket [dc9b1c91db15ae9d|dc9b1c91db]: 3 changes
comment:
  Ticket [dc9b1c91db15ae9d|dc9b1c91db] <i>One trigger may use auxdata belonging to another.</i> status still Closed with 3 other changes


type: ci
user: drh
comment:
  Avoid creating a master journal unless two or more databases in the
  transaction can actually benefit from that master journal.


type: ci
user: drh
comment:
  Always use the sqlite3VdbeDeleteAuxdata() routine for clearing auxdata on
  function parameter, rather than having a separate deleteAuxdataFromFrame()
  for doing the job for trigger frames.


type: ci
user: drh
comment:
  Merge all recent enhancements from trunk, and in particular the
  SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER change.


type: ci
user: dan
comment:
  Fix a problem in the schemalint code that comes up when a sub-query uses one or more of the same tables as its parent.


type: ci
user: drh
comment:
  Remove code not needed now that synchronous=OFF database files no longer
  trigger the use of a master journal.


type: ci
user: drh
ecomment:
  When an index completely covers a table, it can be used as a covering index
  even if columns past the 63rd column are indexed and accessed.

comment:
  When an index complete covers a table, it can be used as a covering index
  even if columns past the 63rd column are indexed and accessed.


type: ci
user: drh
comment:
  In the command-line shell:  When the ".import" command is creating a new
  table using column names from the first row of CSV input, make sure 
  double-quotes in the name are properly escaped.


type: ci
user: drh
comment:
  Improvement on the previous check-in:  disallow automatic indexes for any
  loop that is expected to run less than twice.


type: ci
user: drh
ecomment:
  Query planner improvements:  Take the LIMIT into account when estimated the
  cost of an ORDER BY.  Avoid automatic indexes on query loops that are expected
  to run less than twice.
  See [9e2b2681] for a later enhancement.

comment:
  Query planner improvements:  Take the LIMIT into account when estimated the
  cost of an ORDER BY.  Avoid automatic indexes on query loops that are expected
  to run less than twice.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge recent enhancements from trunk.  Default page size is 4096.  Writes
  to statement journals are avoided.


type: ci
user: drh
comment:
  Change the "PRAGMA stats" command to report size values in LogEst units.
  The eliminates the need for sqlite3LogEstToInt() unless non-standard
  compile-time options are used, so leave it out except in those cases.


type: ci
user: drh
comment:
  Fix an incorrect cost computation for sorting (introduced just a few
  check-ins ago).  Should be NlogN, not NloglogN.


type: ci
user: drh
comment:
  Ephemeral tables used to hold manifested views may not be unordered since
  they might be subject to an ordered comparison if they are part of a join.


type: ci
user: drh
comment:
  Fix a typo in a comment (though an important comment in that it is used 
  to generate documentation).


type: ci
user: drh
comment:
  Provide the new SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER option to 
  sqlite3_db_config() that can be used to activate the two-argument version
  of fts3_tokenizer() for a specific database connection at run-time.


type: ci
user: dan
comment:
  Fix the ICU extension LIKE function so that it does not read past the end of a buffer if it it passed malformed utf-8.


type: ci
user: drh
comment:
  Merge recent enhancements from trunk, and especially the changes that
  reduce the heap-memory footprint of schemas, and defer opening and writing
  to statement journals.


type: ci
user: drh
comment:
  Enhance documentation of sqlite3_snapshot_open() to explain that the database
  connection must have participated in at least one read operation prior to
  the beginning of the transaction for which the snapshot is to be opened.
  Add test cases for this fact.


type: ci
user: drh
comment:
  Enhance the ".stats" command in sqlite3.exe to show one-time stats information
  if invoked with one argument.  Also show /proc/PID/io information if run on
  Linux.


type: ci
user: drh
comment:
  Experimental "PRAGMA onconflict=FAIL" statement to change the default
  ON CONFLICT algorithm to something other than ABORT.


type: ci
user: dan
comment:
  Extend the code in memjournal.c so that it subsumes the role of journal.c. And (untested) can flush journal or statement journal files to disk after they grow to a specified size.


type: ci
user: drh
comment:
  Update the parser so that it pulls out the column name and type all in one
  go, rather than using separate reductions.


type: ci
user: drh
comment:
  Eliminate the need for the Column.zDflt (using Column.pDflt instead) to reduce
  the amount of memory needed to hold the schema.


type: ci
user: drh
comment:
  Very minor improvement to the performance and reduction in size to the
  parser by capturing the name and datatype of table columns in a single
  grammar rule reduction.


type: ci
user: drh
comment:
  Reduce the amount of heap required to store many schemas by storing each
  column datatype appended to the column name, rather than as a separate
  allocation.


type: ci
user: drh
comment:
  Reduce the amount of heap memory required to store many schemas by storing 
  the column datatype appended to the column name (with a \000 separator), 
  rather than in separate memory allocation.


type: ci
user: drh
comment:
  The ANALYZE command automatically appends "noskipscan" to sqlite_stat1 entries
  that have large worst-case repeat estimates but small average repeat estimates.


type: ci
user: dan
comment:
  When using a temporary file for a statement journal, store the first 64KiB in memory. If the file grows larger than that, flush it to disk and free the memory. Hardcoding to 64KiB is just an experiment to check that the memjournal.c code works.


type: ci
user: dan
comment:
  Fix a problem in fts5 where a corrupt db could lead to a (huge) buffer overread. Cherrypick of [c9a30e117f].


type: ci
user: dan
comment:
  Add -DSQLITE_ENABLE_FTS5 to one of the release-test configurations in releasetest.tcl. Run fts5 tests as part of "make test" if SQLITE_ENABLE_FTS5 is defined.


type: ci
user: drh
comment:
  Add the optional -DSERIES_OMIT_CONSTRAINT_VERIFY=0 option to the series.c
  extension that implements the generate_series() virtual table.


type: ci
user: drh
comment:
  Allow the left-hand side of IN operators on virtual tables to have the
  aConstraintUsage[].omit flag clear.


type: ci
user: dan
comment:
  Fix a recently introduced problem in permutations.test causing an error when the QUICKTEST_OMIT environment variable is defined.


type: ci
user: drh
comment:
  Update the configure script to detect pread/pwrite and update os_unix.c to
  use those routines if they are available.


type: g
user: drh
comment:
  Edit [2cffb9e50bed77d1|2cffb9e50b]: Move to branch [/timeline?r=pread&nd&dp=2cffb9e50bed77d1&unhide | pread].


type: ci
user: drh
comment:
  Change the default page size for new database files to 4096 (from 1024).
  Except, when building the testfixture, preserve the legacy page size.
  Also fix a comment on SQLITE_MAX_ATTACHED.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Merge the virtual table query planner enhancement, the RTREE cost estimate
  fix, and the statement journal spill delay enhancement from trunk.


type: ci
user: drh
comment:
  Fix an assert() in sqlite3VarintLen(), even though it is impossible to hit
  in SQLite due to the way sqlite3VarintLen() is used.


type: ci
user: drh
comment:
  Remove the unused journal.c source file (its function have been subsumed into
  memjournal.c).  Refactor some of the names in memjournal.c.  No functional
  changes.


type: ci
user: dan
comment:
  Change the way SQLite invokes the xBestIndex method of virtual tables so that N-way joins involving virtual tables work as expected.


type: ci
user: drh
comment:
  Adjust the cost estimates for RTREE such that the expected number of rows and
  expected cost is halved for each additional constraint.


type: ci
user: dan
comment:
  Fix a problem in the previous commit affecting queries with three or more tables joined together to the right of a LEFT or CROSS JOIN operator.


type: ci
user: dan
comment:
  Update fts3/4 so that the 'merge=X,0' command merges X pages from all segments of the first level in the fts index that contains 2 or more segments.


type: ci
user: drh
comment:
  Avoid a NULL pointer dereference following an OOM while generating code
  for IN operators on virtual tables.


type: ci
user: drh
comment:
  Add a NEVER() to the code generator.  Change the parameter name "mExtra"
  to "mPrereq" in the query planner, to more closely reflect its meaning.


type: ci
user: drh
comment:
  Changes so that some assert()s in the virtual table query planner are
  correct even following an OOM error.


type: ci
user: drh
comment:
  Add compile-time options SQLITE_DEFAULT_SYNCHRONOUS and
  SQLITE_DEFAULT_WAL_SYNCHRONOUS used to specify the default synchronous
  settings for all database connections.


type: ci
user: drh
comment:
  If SQLITE_DEFAULT_WAL_SYNCHRONOUS is not the same value as
  SQLITE_DEFAULT_SYNCHRONOUS and the application has not run
  "PRAGMA synchronous", then set synchronous to the 
  SQLITE_DEFAULT_WAL_SYNCHRONOUS setting when entering WAL mode for
  the first time.


type: ci
user: drh
comment:
  Make the SQLITE_DEFAULT_SYNCHRONOUS and SQLITE_DEFAULT_WAL_SYNCHRONOUS
  values zero-based to agree with PRAGMA synchronous.


type: ci
user: drh
comment:
  Add the SQLITE_DEFAULT_SYNCHRONOUS and SQLITE_DEFAULT_WAL_SYNCHRONOUS
  compile-time options.  Automatically switch to the WAL_SYNCHRONOUS setting
  when first opening a WAL-mode database if the synchronous setting has not
  been previously set by the application.


type: ci
user: dan
comment:
  Update fts3/4 so that the 'merge=X,Y' command merges at least, instead of exactly, Y segments from a single level. This matches the documentation. 'merge=X,0' is, as it was in 3.11, an error.


type: ci
user: drh
comment:
  Drop support for SQLITE_EXTRA_DURABLE.  The new SQLITE_DEFAULT_SYNCHRONOUS
  compile-time option is a more general replacement.


type: ci
user: drh
comment:
  Merge updates from trunk - FTS5 fixes and enhancemenets to the tests scripts
  so that they work with SEE.


type: ci
user: drh
comment:
  In memjournal.c, reuse the same sqlite3_file object for both the in-memory
  phase and the on-disk phase.


type: ci
user: drh
comment:
  Move the write failure test in memjrnlCreateFile() to just after the
  actual write, thus reducing the number of instances of the test by one.


type: ci
user: drh
comment:
  In the query planner, make sure OOM errors are reported up into
  whereLoopAddVirtual() so that it shuts down appropriately.


type: ci
user: drh
comment:
  When optimizing expressions of the form "x IN (SELECT ...)" make sure that
  the subquery is not correlated.  Fix for ticket [5e3c886796e5512].


type: t
user: drh
brief: Fixed ticket [5e3c886796e5512e|5e3c886796].
comment:
  Fixed ticket [5e3c886796e5512e|5e3c886796]: <i>Correlated subquery on the RHS of an IN operator</i> plus 5 other changes


type: ci
user: dan
comment:
  Fix a problem in fts3/4 that was causing it to discard data cached in-memory if an 'optimize' command is run when there is no data on disk. The usual way this would happen is if the very first transaction that writes to the fts3/4 table also includes an 'optimize' command.


type: ci
user: dan
comment:
  Add an incremental optimize capability to fts5. Make the 'merge' command independent of the 'automerge' settings.


type: ci
user: drh
comment:
  Use #ifdefs to remove code that is unreachable in some configurations, replacing
  it with an assert().


type: g
user: drh
comment:
  Edit [f632bba0d6ad3edc|f632bba0d6]: Move to branch [/timeline?r=test-in-subdir&nd&dp=f632bba0d6ad3edc&unhide | test-in-subdir].


type: ci
user: drh
comment:
  More test-case changes so that everything works when the reserved_bytes
  value in the header is non-zero.


type: ci
user: drh
comment:
  Implement FROM-clause subqueries as co-routines whenever they are guaranteed
  to be the outer-most loop of the join.


type: ci
user: drh
comment:
  Change the OP_IntegrityCk opcode to accept the list of root pages as an
  intarray argument in P4.  Slightly smaller and faster.


type: ci
user: drh
comment:
  Fix the query planner so that it is able to use the integer primary key
  from an index as part of a multi-column constraint.


type: ci
user: drh
comment:
  Enhance Lemon so that it reorders the reduce rules such that rules without
  actions occur at the end and so that the first rule is number 0.  This
  reduces the size of the jump table on the reduce switch, and helps the parser
  to run faster.


type: ci
user: dan
comment:
  Have fts5 cache the decoded structure of fts5 indexes in memory. Use "PRAGMA data_version" to detect stale caches.


type: ci
user: drh
comment:
  Change the xGetLastError() method on the windows VFS so that it continues to
  format an error message (if requested) but returns the system error code,
  rather than an SQLite status code.


type: ci
user: drh
comment:
  Changes to releasetest.tcl:  Rename the "OS-X" configuration to "Apple".
  Bring the "Apple" configuration up to date.  Allow the use of "if:os=..."
  arguments in a configuration.  Run the Apple configuration on Linux in addition
  to on Macs, but without -DSQLITE_ENABLE_LOCKING_STYLE.


type: ci
user: dan
comment:
  Add tests for the changes on this branch. Fix a problem with calls to the new progress indicator API made after an rbu update has been resumed.


type: ci
user: dan
comment:
  Change the name of the new API on this branch to "sqlite3_bp_progress". Add tests and documentation for the same.


type: ci
user: dan
comment:
  Add further tests for sqlite3rbu_bp_progress(). Fix a problem in handling WITHOUT ROWID tables in the same.


type: ci
user: dan
comment:
  Add the sqlite3rbu_bp_progress() API to the RBU extension. Used to obtain the percentage progress of an RBU update.


type: ci
user: dan
comment:
  Change the way fts5 internally allocates segment ids in order to eliminated non-determinism from the module.


type: ci
user: drh
comment:
  Make sure system errors that occur durign sqlite3_step() are captured for
  use by sqlite3_system_errno().


type: ci
user: dan
comment:
  Have fts5 cache the structure of its index in main memory. Use "PRAGMA data_version" to figure out when this cache should be invalidated.


type: ci
user: drh
comment:
  Avoid the possibility of integer overflow on a pointer comparison test for
  corruption in the database file.


type: ci
user: drh
comment:
  Create the "uptr" typedef (the same as uintptr_t when available) and use it
  to cast pointers before comparison.


type: ci
user: drh
comment:
  The sqlite3_column_decltype() routine should return NULL, not an empty string,
  if the column has no declared type.


type: ci
user: dan
comment:
  Fix "ifcapable" tests in capi3.test and capi3c.test so that the tests work with more build configurations.


type: ci
user: mistachkin
comment:
  Prevent negative values of SQLITE_DEFAULT_CACHE_SIZE from making SQLITE_WIN32_HEAP_INIT_SIZE negative.


type: ci
user: drh
comment:
  Make test cases in shell1.test robust against TCL deciding to quote strings
  using lots of backslashes.


type: ci
user: drh
comment:
  Add the catchcmd_collapse_space command and use it to help make shell testing
  more robust against TCL space escapes.


type: ci
user: dan
comment:
  Open a statement transaction for "CREATE VIRTUAL TABLE" statements in order to ensure that if the xCreate() call fails, changes made to the sqlite_master and possibly other tables are rolled back.


type: ci
user: drh
comment:
  Fix the multiplexor so that it does not assume that the xGetLastError method
  is non-NULL in the child VFS.


type: ci
user: dan
comment:
  Add further tests for savepoint rollback. Fix various code issues and add missing comments in fts5_index.c.


type: ci
bgcolor: #b1d3a8
user: drh
comment:
  Avoid compiler warnings and excess opcodes when SQLITE_ENABLE_PREUPDATE_HOOK
  is not used.  Fix the EXPLAIN output for P4_TABLE opcodes.


type: ci
user: drh
comment:
  Improvements to the performance testing program "speedtest1.c" and the
  shell script that invokes it.


type: ci
user: drh
comment:
  Simplifications and clarification to update callback handling in the
  OP_Delete and OP_Insert opcodes.


type: ci
user: dan
comment:
  Have the sqlite3session_apply() function and its streaming equivalent retry any operations that failed with SQLITE_CONSTRAINT after all other operations on the same table have been attempted. New code is largely untested.


type: ci
user: drh
comment:
  Enhance sqlite3session_apply() and sqlite3session_apply_strm() so that
  conflicts are retried before the xConflict() callback is invoked, as long
  as the "apply" operation is making forward progress.


type: ci
user: drh
comment:
  Enhance the query planner so that IS and IS NULL operators are able to drive
  an index on a LEFT OUTER JOIN.


type: ci
user: drh
comment:
  When reading an sqlite_stat1 table with missing entries for some indexes,
  set the default for the missing indexes after all other indexes are set
  so that the default values are more in line with indexes that are actually
  in the stat1 table.


type: ci
user: dan
comment:
  Test that the view name is passed to the authorization callback when a SELECT statement is run on a view.


type: ci
user: drh
comment:
  Add the SQLITE_DBCONFIG_REQUIRE_TXN argument for sqlite3_dbconfig() which when
  set requires an explicit transaction before updating the database.


type: ci
user: drh
comment:
  Make sure "0" intended as a NULL pointer is cast to a pointer when used
  in a varargs functions parameter.


type: ci
user: drh
comment:
  Incorrect version of build.c checked in at [87e5f5a6c60e3] (because I
  neglected to press the "Save" button on the text editor).  There is no change
  to the logic, just improved presentation.


type: ci
user: drh
comment:
  Attempt to treat the arguments to a table-valued function as if they occur
  in the ON clause of a LEFT JOIN rather than in the WHERE clause.  But this
  causes undesirable behavior with generate_series, as demonstrated by test
  cases.  This is an incremental check-in pending further work.


type: g
user: drh
comment:
  Edit [8f1f300c284ea1e4|8f1f300c28]: Move to branch [/timeline?r=mistake&nd&dp=8f1f300c284ea1e4&unhide | mistake]. Mark "Closed". Edit check-in comment. Change user to "drh".


type: ci
user: dan
comment:
  Defer opening the file used for the temp database (where CREATE TEMP TABLE tables are stored) until the database is too large to reside entirely within the cache. There are likely still problems on this branch.


type: ci
user: dan
comment:
  For a pager backed by a temp file, store the main journal in memory until it is at least sqlite3_config.nStmtSpill bytes in size. Prevent the backup API from being used to change the page-size of a temp file.


type: ci
bgcolor: #80bbd6
user: drh
comment:
  Prevent the in-memory journal read cursor from entering an inconsistent state
  when it reads the last few bytes out of the journal file.  Fix for
  ticket [7f7f8026eda38].


type: t
user: drh
brief: Ticket [7f7f8026eda387d5|7f7f8026ed]: 6 changes
comment:
  Ticket [7f7f8026eda387d5|7f7f8026ed] <i>Segfault following heavy SAVEPOINT usage</i> status still Open with 6 other changes


type: t
user: drh
brief: Ticket [7f7f8026eda387d5|7f7f8026ed]: 3 changes
comment:
  Ticket [7f7f8026eda387d5|7f7f8026ed] <i>Segfault following heavy SAVEPOINT usage</i> status still Open with 3 other changes


type: t
user: drh
brief: Fixed ticket [7f7f8026eda387d5|7f7f8026ed].
comment:
  Fixed ticket [7f7f8026eda387d5|7f7f8026ed]: <i>Segfault following heavy SAVEPOINT usage</i> plus 3 other changes


type: ci
user: drh
ecomment: Faulty check-in. Extraneous edits that were not logically part of the check-in.
comment:
  Remove quotes from identifiers used as datatype names in a CREATE TABLE
  statement.  Fix for ticket [7d7525cb01b68]


type: ci
user: drh
comment:
  Fix the sqlite3BtreeDelete() routine so that it preserves the correct
  key even when the row being deleted is not on a leaf page.
  Fix for ticket [a306e56ff68b8fa56]


type: t
user: drh
brief: New ticket [a306e56ff68b8fa5|a306e56ff6].
comment:
  New ticket [a306e56ff68b8fa5|a306e56ff6] <i>Failed DELETE with PRAGMA reverse_unordered_selects</i>.


type: ci
user: drh
comment:
  Fix a problem in the code generator for joins on virtual tables where the
  outer loop of the join uses the IN operator.


type: t
user: drh
brief: Fixed ticket [a306e56ff68b8fa5|a306e56ff6].
comment:
  Fixed ticket [a306e56ff68b8fa5|a306e56ff6]: <i>Failed DELETE with PRAGMA reverse_unordered_selects</i> plus 5 other changes


type: ci
user: drh
comment:
  Factor out the common operation of setting the Expr.x.pSelect field of an
  Expr object into a subroutine.


type: ci
user: drh
comment:
  Back off of the parser optimization in the previous check-in, slightly, to
  preserve some backwards compatibility regarding some undocumented behavior
  in the '#AAA' style query parameter.


type: ci
user: drh
comment:
  Remove the VdbeOp.opflags field and its associated initialization overhead.
  Update mkopcodeh.tcl to reorder opcode numbers to help the resolveP2Values()
  routine run faster.


type: ci
user: drh
comment:
  Simplification and performance improvements in sqlite3ExprDup() and its
  subroutines.  More work is possible in this area.


type: ci
user: drh
comment:
  Performance optimizations in the column cache of the code generator, and
  especially the sqlite3ExprCacheRemove() routine.


type: ci
user: dan
comment:
  Update the documentation for sqlite3_snapshot_cmp() to make the circumstances under which the comparison is valid clearer. Add tests for the same.


type: ci
user: dan
comment:
  Once a temporary database file has been opened, flush all dirty pages to disk when comitting a transaction.


type: ci
user: drh
comment:
  Add the --indent option to the .schema and .fullschema commands in the shell,
  to enable simple but effective pretty-printing.


type: ci
user: dan
comment:
  On commit, flush dirty temp-file pages only if the file is already open and 25% or more of the cache is dirty.


type: ci
user: drh
comment:
  Additional ".selecttrace" debugging output when the query is transformed from a
  DISTINCT into a GROUP BY.


type: ci
user: drh
comment:
  Fix compiler warnings by removing surplus local variables in the schema
  pretty-printer logic of the command-line shell.


type: ci
user: drh
comment:
  Since the return value of sqlite3OsClose() is always ignored, we might as
  well make it "void" instead of "int", and thereby save 50 bytes in the
  compiled binary.


type: ci
user: dan
comment:
  Correctly interpret negative "PRAGMA cache_size" values when determining the cache-size used for sorting large amounts of data (i.e. the functionality in vdbesort.c).


type: ci
user: drh
comment:
  Add the SF_HasAgg constant (currently unused).  Also enhance the comments on
  many other constant definitions to detail constraints on their values.


type: ci
user: dan
comment:
  In the ICU extension toupper() and tolower() SQL functions, avoid calling u_strToUpper() or u_strToLower() a second time if the buffer passed to the first invocation turns out to be large enough.


type: ci
user: drh
comment:
  CLI enhancement:  Add the ".eqp full" option, that shows both the EXPLAIN
  QUERY PLAN and the EXPLAIN output for each command run.  Also disable
  any ".wheretrace" and ".selecttrace" when showing EQP output.


type: ci
user: drh
comment:
  Progress toward getting DISTINCT to use a seek to advance to the next distinct
  row, when driven by an appropriate index.


type: ci
user: drh
comment:
  Skip-ahead is now just an optimization.  If it gets confused, it falls back
  to an incremental scan with redundancy elimination.


type: ci
user: dan
comment:
  Add the experimental sqlite3rbu_vacuum() API function. For opening an RBU handle that rebuilds a database from scratch.


type: ci
user: dan
comment:
  Avoid creating any extra files ("<target>-vacuum") when running an RBU vacuum. Ensure that the OAL file created is "<target>-oal", not "<target>-vacuum-oal".


type: t
user: drh
brief: New ticket [7d7525cb01b68712|7d7525cb01].
comment:
  New ticket [7d7525cb01b68712|7d7525cb01] <i>Compatibility break on INTEGER PRIMARY KEY columns.</i>.


type: ci
user: drh
comment:
  <b>API Change:</b> Modify sqlite3_enable_load_extension() so that it only
  enables/disables the load_extension() SQL function, and leaves the C-APIs
  enabled at all times.  In this way, applications can enable extension loading
  for the C interface without having to expose that capability to the SQL.


type: g
user: drh
comment:
  Edit [abf9d602eefd9de6|abf9d602ee]: Move to branch [/timeline?r=mistake&nd&dp=abf9d602eefd9de6&unhide | mistake]. Add propagating "hidden". Mark "Closed". Edit check-in comment.


type: t
user: drh
brief: Fixed ticket [7d7525cb01b68712|7d7525cb01].
comment:
  Fixed ticket [7d7525cb01b68712|7d7525cb01]: <i>Compatibility break on INTEGER PRIMARY KEY columns.</i> plus 5 other changes


type: g
user: drh
comment:
  Edit [92dc59fd5ad66f64|92dc59fd5a]: Change background color to "#d0c0ff". Edit check-in comment. Add tag "version-3.12.2".


type: ci
user: dan
comment:
  Detect attempts to use rbu vacuum on a wal mode database (not allowed). And attempts to write to a database in the middle of an rbu vacuum (which prevents the vacuum from resuming).


type: ci
user: dan
comment:
  When an RBU vacuum is started on a db identified using a URI filename, pass the same URI parameters when creating the new version of the db. This ensures that RBU vacuum works with password protected databases.


type: ci
user: dan
comment:
  Updates to ensure the values of PRAGMA settings like "page_size", "auto_vacuum", "user_version" and "application_id" are not lost when a database is RBU vacuumed.


type: ci
user: drh
comment:
  Revert sqlite3_enable_load_extension() to its original long-standing behavior.
  Add SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION which will enable only the C-API and
  leave the SQL function disabled.


type: ci
user: drh
comment:
  When an error occurs while transitioning out of WAL mode, make sure the
  locking state is not left at EXCLUSIVE.


type: ci
user: drh
comment:
  Fix a locking race condition in Windows 10 that can occur when two or more
  processes attempt to recover the same hot journal at the same time.


type: ci
user: drh
comment:
  Add the SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION method for enabling 
  sqlite3_load_extension() while leaving the load_extension() SQL function
  disabled.


type: ci
user: dan
comment:
  If a call to sqlite3rbu_close() on an rbu handle opened by sqlite3rbu_vacuum() returns other than SQLITE_OK, delete the contents of the rbu_state table. This ensures that if an RBU vacuum operation either completes successfully or encounters an error, the next call to sqlite3rbu_vacuum() with the same parameters starts a new RBU vacuum operation.


type: ci
user: dan
comment:
  Add the sqlite3rbu_vacuum() API, to create an RBU handle that can be used to rebuild a database in the same way as the VACUUM command.


type: ci
user: drh
comment:
  Change the temporary directory search algorithm on unix so that directories
  with only -wx permission are allowed.  And do not allow "." to be returned if
  it lacks -wx permission.


type: ci
user: drh
comment:
  Fix the fix to the temporary directory search algorithm so that it continues
  to return "." as a fallback if that directory has the correct permissions.


type: ci
user: drh
comment:
  When checking for the WHERE-clause push-down optimization, verify that
  all terms of the compound inner SELECT are non-aggregate, not just the
  last term.  Fix for ticket [f7f8c97e97597].


type: t
user: drh
brief: New ticket [f7f8c97e975978d4|f7f8c97e97].
comment:
  New ticket [f7f8c97e975978d4|f7f8c97e97] <i>Valid query fails to compile due to WHERE clause optimization</i>.


type: t
user: drh
brief: Fixed ticket [f7f8c97e975978d4|f7f8c97e97].
comment:
  Fixed ticket [f7f8c97e975978d4|f7f8c97e97]: <i>Valid query fails to compile due to WHERE clause optimization</i> plus 5 other changes


type: ci
user: drh
comment:
  Fix the "checksymbols" target in Makefile.in to be able to deal with the
  sqlite3changegroup family of interfaces.


type: ci
user: dan
comment:
  Fix a couple of test script problems on this branch. Both related to the fact that temp database page sizes can no longer be changed by VACUUM or the backup API after the temp db is populated.


type: ci
user: drh
comment:
  More simplification of the sqlite3AtoF() routine.  Add special comments
  to indicate branches that are for optimization purposes only and that give
  the correct answer even if always or never taken.


type: ci
user: dan
comment:
  Modify the permutations.test script so as to set any permutation specific configuration values before running each individual test script.


type: ci
user: drh
comment:
  Change the sqlite3Atoi64() routine so that it returns failure if not all of
  the input characters are consumed, even if it consumed all characters up to
  the first 0x00.  This has no impact on external APIs as far as I can tell.


type: ci
user: dan
comment:
  Avoid running sessionfault.test and sessionfault2.test as part of the "inmemory_journal" permutation.


type: ci
user: drh
comment:
  Rearrange some code in the RowSet logic for clarity of presentation, while
  adding an /*OPTIMIZATION-IF-TRUE*/ comment.  It should operate identically.


type: ci
user: drh
comment:
  Comment changes only:  Add several optimization marks in rowset.c.  Add
  a header comment that explains what the various special comments mean.


type: ci
user: drh
comment:
  Further simplifications and improved commentting on the rowset.c module,
  including several optimization comments.


type: ci
user: drh
comment:
  Lemon bug fix:  Do not merge action routines unless their destructors are
  also identical.  Problem and suggested fix reported on the mailing list
  by Kelvin Sherlock.


type: ci
user: dan
comment:
  Modify the permutations.test script so as to set any permutation specific configuration values before running each individual test script. Fix a mostly harmless buffer overread in the sessions module.


type: ci
user: dan
comment:
  Fix test script temptable2.test so that it works with the "inmemory_journal" and "journaltest" permutations.


type: ci
user: drh
comment:
  Postpone I/O associated with TEMP files for as long as possible, with the
  hope that the I/O can ultimately be avoided completely.


type: ci
user: drh
comment:
  Fix the temporary directory search algorithm for unix so that it fails
  gracefully even if all candidate directories are inaccessible.  This fixes
  a bug that was introduced by check-in [9b8fec60d8e].


type: ci
user: dan
comment:
  Change the way tester.tcl handes $argv so that in "permutations.test <permutation> <filename>" <filename> may be the name of any file in the test/ directory.


type: ci
user: drh
comment:
  Improvements to the way the LIKE optimization is implemented, resulting in
  slightly smaller and faster code that is easier to test.


type: ci
user: drh
comment:
  Disable the multi-row one-pass DELETE optimization when the WHERE clause
  contains a subquery.  Fix for ticket [dc6ebeda9396087].


type: t
user: drh
brief: Fixed ticket [dc6ebeda93960877|dc6ebeda93].
comment:
  Fixed ticket [dc6ebeda93960877|dc6ebeda93]: <i>Incorrect DELETE due to the one-pass optimization</i> plus 6 other changes


type: ci
user: drh
comment:
  Fix a test case in temptable2.test so that it works on systems that
  lack a coherient cache (ex: OpenBSD) and thus do not support the
  "PRAGMA mmap_size" command.


type: ci
user: drh
comment:
  Turn the session extension off by default.  Enable using --enable-session on
  configure scripts, or using the SESSION=1 argument to nmake on Windows.


type: ci
user: drh
comment:
  The session extension is disabled by default.  To enable it using
  --enable-session on ./configure on unix and add SESSION=1 to the nmake
  on Windows.  Or add -DSQLITE_ENABLE_SESSION and
  -DSQLITE_ENABLE_PREUPDATE_HOOK to build manually.


type: ci
user: dan
comment:
  Remove a redundant condition from pcache.c. Add an OPTIMIZATION-IF-TRUE comment to another condition that requires it.


type: ci
user: dan
comment:
  Disable the PagerDontWrite() optimization for temp tables. It can cause database corruption if a page passed to PagerDontWrite() is dirty at the start of a transaction that is subsequently rolled back.


type: ci
user: drh
comment:
  Only disable the one-pass DELETE optimization if the WHERE clause contains
  a correlated subquery.  Uncorrelated subqueries are allowed.  This is a
  refinement of check-in [3f221f592a9a1] that is the fix for
  ticket [dc6ebeda9396087].


type: ci
user: drh
comment:
  Initial check-in of the "scrub.exe" utility program prototype.  Not yet
  fully functional.  In particular, no scrubbing is done.


type: ci
user: drh
comment:
  Additional error reporting.  Open the source database read/write so that
  it can delete the WAL file when done.


type: t
user: drh
brief: Ticket [16c9801ceba49239|16c9801ceb]: 6 changes
comment:
  Ticket [16c9801ceba49239|16c9801ceb] <i>Assertion fault on DELETE</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  In the WHERE generator, when building code for a DELETE operation, make sure
  that seeks to the main table are not deferred.
  This is a better fix for the [16c9801ceba49] bug than the previous.


type: ci
user: drh
comment:
  Do not scan indexes that have the integer primary key as their left-most
  column.  This fixes the crash of ticket [16c9801ceba] but it seems like
  the wrong fix.  More investigation needed.


type: t
user: drh
brief: Ticket [16c9801ceba49239|16c9801ceb]: 3 changes
comment:
  Ticket [16c9801ceba49239|16c9801ceb] <i>Assertion fault on DELETE</i> status still Open with 3 other changes


type: ci
user: dan
comment:
  Fix an error in main.mk causing testfixture to use individual fts5 source code files instead of fts5.c.


type: ci
user: dan
comment:
  Fix a bug in sqlite_analyzer causing it report (slightly) incorrect values for the number of entries in indexes or WITHOUT ROWID tables.


type: t
user: drh
brief: Ticket [16c9801ceba49239|16c9801ceb]: 3 changes
comment:
  Ticket [16c9801ceba49239|16c9801ceb] <i>Segfault on DELETE with WHERE containing OR</i> status still Open with 3 other changes


type: ci
user: drh
comment:
  For DELETE operations, make sure that seeks on the main table are not
  deferred, since if they are and none of the indexes reference columns of
  the table, the seek might never occur until the OP_Delete opcode, which is
  too late.  Fix for ticket [16c9801ceba49].


type: t
user: drh
brief: Fixed ticket [16c9801ceba49239|16c9801ceb].
comment:
  Fixed ticket [16c9801ceba49239|16c9801ceb]: <i>Segfault on DELETE with WHERE containing OR</i> plus 3 other changes


type: ci
user: drh
comment:
  Simplification to the logic used to decide between OP_Seek and OP_NotExists.
  Use OP_NotExists for both DELETE and UPDATE.


type: ci
user: dan
comment:
  Add test/bc_test1.c, for testing the degree of concurrency provided by this branch under various conditions.


type: ci
user: drh
comment:
  Fix a problem in sqlite3PagerMovepage() when working on a temp table for which
  pages have been spilled.


type: ci
user: drh
comment:
  Add pcache tracing macros.  Off by default.  Requires changing an "#if 0"
  and recompiling to enable.


type: ci
user: drh
comment:
  Do not reset the page cache when a rollback journal truncate fails, as doing
  so can cause loss of information for an in-memory TEMP file.


type: ci
user: drh
comment:
  Enhance the sqlite3_load_extension() API so that the first parameter (the "db"
  parameter) can be NULL.  An extension that is not associated with any 
  database connection remains loaded for the life of the process.


type: g
user: dan
comment:
  Edit [4533ed352db2e05c|4533ed352d]: Move to branch [/timeline?r=mistake&nd&dp=4533ed352db2e05c&unhide | mistake]. Mark "Closed".


type: ci
user: drh
comment:
  Fix pager_end_transaction() to ensure that a ROLLBACK does not clobber
  unwritten pages in a TEMP file pcache.


type: ci
user: drh
comment:
  Remove the unused PGHDR_NEED_READ flag.  Add invariant checking (during
  SQLITE_DEBUG builds only) for the PgHdr object.


type: ci
user: drh
comment:
  Enhance the scrub utility program so that it does a FULL checkpoint prior to
  starting the backup, to ensure that the database file content matches what needs
  to be backed up without having to look at the WAL file.


type: ci
user: drh
comment:
  Add the "scrub" utility program that simultaneously backs-up a live database
  and erases all deleted content.


type: ci
user: drh
comment:
  Clean up the WHERE_* macros used for the wctrlFlags parameter on the 
  sqlite3WhereBegin() interface, freeing up some bits to be used for other things.


type: ci
user: drh
comment:
  In a query with both ORDER BY and LIMIT, if the inner loop satisfies the 
  ORDER BY then try to cut short each invocation of the inner loop once the
  LIMIT has been satisfied.  This check-in is a partial implementation only.


type: ci
user: drh
comment:
  Set the NULLEQ flag on the sequence counter comparison in the ORDER BY LIMIT
  optimization, to avoid coverage complaints about not testing the NULL case.


type: ci
user: drh
comment:
  For queries with both ORDER BY and LIMIT, if the rows of the inner loop are
  emitted in ORDER BY order and the LIMIT has been reached, then optimize by
  exiting the inner loop and continuing with the next cycle of the first outer
  loop.


type: ci
user: drh
comment:
  Autoconf configure.ac adjustment to try to get it to look for both editline
  and readline automatically.


type: ci
user: drh
comment:
  Optimizations to link list merge sort code in vdbesort.c, pcache.c, and
  rowset.c.  Resulting binaries are 10 bytes smaller and use 0.03% fewer CPU
  cycles.


type: ci
user: drh
comment:
  Simplify the sqlite3BtreeInsert() interface by gathering the five arguments
  describing the content to be inserted into the new BtreePayload structure, and
  thus reducing the number of parameters from eight to four.


type: ci
user: drh
comment:
  Update the configure script with additional hints on the location of
  tclConfig.sh, hints needed by the latest versions of Xcode.


type: ci
user: drh
comment:
  Lemon enhancement: avoid unnecessary reduce actions that convert one
  non-terminal into another but have no side effects.


type: g
user: drh
comment:
  Edit [bfbb6dd84b5e1335|bfbb6dd84b]: Move to branch [/timeline?r=tree-stat1-fix&nd&dp=bfbb6dd84b5e1335&unhide | tree-stat1-fix].


type: ci
user: drh
comment:
  Use a pointer to the top of the stack rather than an index into the stack
  in the Lemon-generated parser template, for about 6.6% parser performance gain.


type: ci
user: drh
comment:
  Improvements to the initialization of the push-down automoton for the
  Lemon-generated parser.  Smaller and faster.


type: ci
user: dan
comment:
  Fix an obscure problem with transactions written in "PRAGMA synchronous=full" mode on systems that do not support POWERSAFE_OVERWRITE causing an xSync() call to be omitted if the last frame written by a transaction is aligned to a sector boundary. This means that if a power failure or OS crash occurs very soon after such a transaction is committed, it may be lost following system recovery.


type: ci
user: drh
comment:
  Minor change to the walcrash4.test module so that it works when
  SQLITE_DEFAULT_WAL_SYNCHRONOUS is set to something other than 2.


type: ci
user: drh
comment:
  Enhance Lemon and the parser template so that it can once again build parsers
  that have no unreachable branches.


type: ci
user: drh
comment:
  Add a new OP_SeekRowid opcode, that combines the functions of OP_MustBeInt
  and OP_NotExists.  This makes the code slightly smaller and faster.


type: ci
user: drh
comment:
  Fix the Lemon-generated parser so that it compiles with -DYYSTACKDEPTH=0.
  It does compile now, but there are subtle issues still.


type: ci
user: drh
comment:
  Improvements to WHERE-clause debug tracing.  Show TK_MATCH expressions and
  show more details on WhereTerm traces.


type: ci
user: drh
comment:
  Enhance "PRAGMA compile_options" so that it shows the version of the compiler
  used to generate the executable, for common compilers.


type: ci
user: drh
comment:
  Add the vfsstat.c loadable extension - a VFS shim that measures the amount
  of I/O, and an eponymous virtual table that is used to extract and view
  the measurements.


type: ci
user: drh
comment:
  Provide for a separate default page size for TEMP and transient database
  files and initialize that separate default page size to 1024 instead of 4096.


type: ci
user: dan
comment:
  Update the amalgamation-tarball configure script so that it can use header file "readline/readline.h" with library file "libedit".


type: ci
user: drh
comment:
  Enhance the sqlite3_load_extension() interface to permit extensions to
  return SQLITE_OK_LOAD_PERMANENTLY which will prevent the extensions from
  unloading when the database connection closes.


type: ci
user: drh
comment:
  Enhance the sqlite3_load_extension() interface to permit extensions to 
  return SQLITE_OK_LOAD_PERMANENTLY which will prevents unloading when
  the database connection closes.


type: ci
user: dan
comment:
  Experimental change to allow virtual tables to take advantage of LIKE, REGEXP and GLOB terms that are part of OR expressions within WHERE clauses.


type: ci
user: drh
comment:
  The OR optimization is usable on virtual tables with LIKE, REGEXP and/or GLOB
  terms in the WHERE clause.


type: ci
user: dan
comment:
  Fix an FTS5 problem (segfault or incorrect query results) with "... MATCH 'x OR y' ORDER BY rank" queries when either token 'x' or 'y' is completely absent from the dataset.


type: ci
user: drh
comment:
  An experimental branch with code that allows virtual tables to be declared
  as WITHOUT ROWID tables. This might be useful for virtual tables that model
  external data sources that do not have a convenient way of computing a unique
  rowid.  The current check-in almost works, but there are still serious issues.


type: ci
user: drh
comment:
  Add the CSV extension to the test fixture.  Fix a memory leak in the CSV
  extension.  Add test cases for the CSV extension, including one that uses
  a WITHOUT ROWID virtual table participating in the OR optimization.


type: ci
user: drh
comment:
  Performance optimizations on the CSV virtual table.  Disallow WITHOUT ROWID
  virtual tables that have an xUpdate method, for now.


type: ci
user: drh
comment:
  Add support for virtual tables using a WITHOUT ROWID schema.  This merge also
  includes enhancements to the CSV extension, which is used for testing of
  the new WITHOUT ROWID virtual table mechanism.


type: ci
user: drh
comment:
  Fix up speedtest1.c so that it will compile and run on SQLite versions prior
  to 3.6.18 (circa 2009-09-11).


type: ci
user: drh
comment:
  Change the sqlite3BtreeKeySize() interface into sqlite3BtreeIntegerKey() and
  make it only work for table btrees.  Change sqlite3BtreeDataSize() into
  sqlite3BtreePayloadSize() and make it work for all btrees.  Combine
  sqlite3BtreeDataFetch() and sqlite3BtreeKeyFetch() into a single
  sqlite3BtreePayloadFetch() routine.  These changes seem to make the
  b-tree interface more rational and they reduce both binary size and
  CPU usage.


type: ci
user: drh
comment:
  Btree interface refactoring:
  (1) The sqlite3BtreeKeySize() interface is renamed to sqlite3BtreeIntegerKey()
  and modified to work only for table btrees with a rowid.
  (2) The sqlite3BtreeDataSize() interface is renamed to sqlite3BtreePayloadSize()
  and modified to work with any btree.
  (3) The sqlite3BtreeDataFetch() and sqlite3BtreeKeyFetch() routines are
  combined into a single sqlite3BtreePayloadFetch() routine.
  The result of these changes is a smaller binary and fewer CPU cycles needed
  to run queries.


type: ci
user: drh
comment:
  Initialize the yyerrcnt variable in the lemon parser template.  This has no
  effect on SQLite itself.


type: ci
user: drh
comment:
  Fix lempar.c so that the shift-reduce optimization works for error processing.
  This is a Lemon issue only and has no impact on SQLite.


type: ci
user: drh
comment:
  Translate filenames of the form "/c:/*" into just "c:/*" on WinRT and Cygwin.
  (SQLite has long done this for Win32/WinNT.)


type: ci
user: drh
comment:
  Fix the dbhash utility so that it ignores the root page number when hashing
  the sqlite_master table.  Add new command-line options.  Add the ability to
  hash multiple databases with a single command.


type: ci
user: drh
comment:
  Add the "dbhash.exe" utility program that computes a SHA1 hash over the
  invariant content of an SQLite database file.  Free space in the file, the
  page size, auto_vacuum status, text encoding, and so forth do not change the
  hash.  Only the content matters.


type: ci
user: drh
comment:
  Fix the "onecolumn" and "exists" methods of the TCL interface so that they
  work in combination with the "profile" callback.


type: ci
user: drh
comment:
  Fix RBU so that it builds with -DSQLITE_ENABLE_8_3_NAMES.
  Fix "PRAGMA compile_options" for SQLITE_ENABLE_8_3_NAMES such that it reports
  the numeric setting: "1" or "2".


type: ci
user: dan
comment:
  Add a missing OP_ColumnsUsed opcode to code for expressions like "? IN (SELECT ...)" in cases where expression can use an index that may contain NULL values.


type: ci
user: dan
comment:
  If a table is on the rhs of a LEFT JOIN, include only terms from the joins ON(...) clause in the cursor-hint passed via OP_CursorHint.


type: ci
user: dan
comment:
  Include WHERE terms in the cursor-hint passed to a cursor opened for the rhs of a LEFT JOIN iff we can be sure that those terms will not evaluate to true if the LEFT JOIN generates a row of NULLs.


type: ci
user: dan
comment:
  Allow LIKE operators that appear in a WHERE clause to be included in the cursor-hint for a cursor on the rhs of a LEFT JOIN.


type: ci
user: dan
comment:
  For a table on the rhs of a LEFT JOIN operator, do not include terms like "IS NULL" from the WHERE clause in the cursor-hint. These may be false for rows that the cursor would otherwise visit, but true for a row of all NULL values generated by the LEFT JOIN.


type: ci
user: dan
comment:
  Update the sqldiff utility so that if the --vtab switch is specified "rtree", "fts3", "fts4" and "fts5" tables are diff'd directly and the underlying real database tables ignored. Without this switch, all virtual tables are ignored and the diff is performed on the underlying real tables.


type: ci
user: drh
comment:
  Fix the handling of OP_Eq opcodes that compare a register against itself
  and that require an affinity change.


type: ci
user: dan
comment:
  Add the sqlite3rbu_state() API. Used to determine the current state (creating OAL, ready to move OAL, incremental-checkpoint, finished or error) of an RBU operation.


type: ci
user: drh
comment:
  Fix the transitive constraint logic error that can result in a null pointer
  dereference.  Fix for ticket [e8d439c77685eca6].


type: t
user: drh
brief: New ticket [e8d439c77685eca6|e8d439c776].
comment:
  New ticket [e8d439c77685eca6|e8d439c776] <i>Crash on transitive WHERE constraint and undefined collation</i>.


type: t
user: drh
brief: Fixed ticket [e8d439c77685eca6|e8d439c776].
comment:
  Fixed ticket [e8d439c77685eca6|e8d439c776]: <i>Crash on transitive WHERE constraint and undefined collation</i> plus 6 other changes


type: ci
user: drh
comment:
  Fix a problem in table-valued functions on the RHS of an IN operator that
  occurs following an OOM error.


type: ci
user: drh
comment:
  Fix an off-by-one comparison in the intarray() virtual table.  Get the
  intarray() virtual table tests working using the legacy makefile.


type: ci
user: drh
comment:
  Change the name of the intarray() extension to carray() and give it an
  optional third parameter that specifies the datatype as one of 'int32',
  'int64', 'double', or 'char*'.  'int32' is the default.


type: ci
user: dan
comment:
  Change the name of the SQLITE_DBSTATUS_PROPORTIONAL_CACHE_USED to SQLITE_DBSTATUS_CACHE_USED_SHARED.


type: ci
user: dan
comment:
  Add the SQLITE_DBSTATUS_CACHE_USED_SHARED sqlite3_db_status() parameter. This option is similar to DBSTATUS_CACHE_USED, except that it divides memory used by shared caches evenly between all connections.


type: ci
user: drh
comment:
  In the sqldiff program, make sure the first argument to the db_prepare()
  utility function is always a valid format string.


type: ci
user: drh
comment:
  Fix various internal #defines to conform to new C-language naming restrictions,
  specifically that private-use macros names must not begin with "_".


type: ci
user: drh
comment:
  Add the "#/value-list/" style of results for approximate value matching
  in the do_test command of the test infrastructure.  Use this new result style
  to make the SQLITE_DBSTATUS_CACHE_SIZE_SHARED tests cross-platform.


type: ci
user: dan
comment:
  Add some support for using row value constructors in certain parts of SQL expressions. There are many bugs on this branch.


type: ci
user: drh
comment:
  Fix header comments and remove an unnecessary version restriction from the
  carray() table-valued function implementation.


type: ci
user: drh
comment:
  Interface design for a new sqlite3_trace_v2() method that supersedes
  sqlite3_trace() and sqlite3_profile().


type: ci
user: drh
comment:
  Add the sqlite3_expanded_sql() interface.  Refinements to the
  sqlite3_trace_v2() interface to make it more useful.


type: ci
user: drh
comment:
  Fix ALTER TABLE so that it does not promote the schema version past 3, as
  that will cause DESC indexes to go corrupt.
  Ticket [f68bf68513a1c].


type: t
user: drh
brief: Fixed ticket [f68bf68513a1c15f|f68bf68513].
comment:
  Fixed ticket [f68bf68513a1c15f|f68bf68513]: <i>ALTER TABLE corrupts DESC indexes</i> plus 5 other changes


type: ci
user: dan
comment:
  Temporarily add extra sqlite3_log() calls to this version to help with debugging a performance problem.


type: ci
user: drh
comment:
  Experimental patch restore the slight bias in btree-balancing immediately
  after the change to reduces the amount of memcpy() work done by balancing.


type: ci
user: drh
comment:
  The sqlite3_expanded_sql() function compiles, but always returns NULL, when
  the SQLITE_OMIT_TRACE compile-time option is used.


type: ci
user: dan
comment:
  Add new test file vacuummem.test. To test that any temporary memory used by VACUUM is freed as soon as the VACUUM has finished (not, for example, when sqlite3_close() is finally called).


type: ci
user: dan
comment:
  Add the largely untested SQLITE_FTS5_NO_WITHOUT_ROWID compile time option to fts5. For building a dynamically loadable extension that does not use WITHOUT ROWID.


type: ci
user: dan
comment:
  Allow vector IN(SELECT ...) expressions to use an index if either all the indexed columns are declared NOT NULL or if there is no difference between the expression evaluating to 0 and NULL (as in a WHERE clause).


type: ci
user: drh
comment:
  Be sure to hold the database connection mutex while calling sqlite3VdbeExpand()
  from within sqlite3_expanded_sql().


type: ci
user: mistachkin
comment:
  Revise a warning fix from the previous check-in to improve clarity.  Also, fix an incorrect test prefix (typo).


type: ci
user: drh
comment:
  Improvements to sqlite3_trace_v2() documentation.  Fix the
  sqlite3VdbeExpandSql() routine to respond better to OOM conditions.


type: ci
user: dan
comment:
  Fix where.c handling of "IN (SELECT ...)" expressions when the SELECT returns more than one result column. Also error handling for other row value constructor cases.


type: ci
user: drh
comment:
  Fix code so that the deprecated sqlite3_trace() and sqlite3_profile() 
  interfaces are not called when SQLITE_OMIT_DEPRECATED is used.


type: ci
user: drh
comment:
  Change SQLITE_TRACE_STMT to return unexpanded SQL text in the X argument.
  Add documentation on how and when to use sqlite3_expanded_sql(P) to compute
  the expanded text for legacy sqlite3_trace() compatibility.


type: ci
user: drh
comment:
  Disable the DBSTATUS_CACHE_USED_SHARED tests on mac when memsys3/5 is disabled
  due to differences in the allocation sizes for the Darwin memory allocator.


type: ci
user: drh
comment:
  In the VACUUM command, set the cache_size of the transient vacuum_db database
  to be the same as the database being vacuumed.


type: ci
user: drh
comment:
  Make sure affinity is applied correctly on lower-bound range constraints
  in the min() optimization.  Fix for ticket [a0bac8b3c3d1bb].


type: t
user: drh
brief: Ticket [a0bac8b3c3d1bb75|a0bac8b3c3]: 6 changes
comment:
  Ticket [a0bac8b3c3d1bb75|a0bac8b3c3] <i>CREATE INDEX changes the answer</i> status still Open with 6 other changes


type: t
user: drh
brief: Fixed ticket [a0bac8b3c3d1bb75|a0bac8b3c3].
comment:
  Fixed ticket [a0bac8b3c3d1bb75|a0bac8b3c3]: <i>CREATE INDEX changes the answer</i> plus 3 other changes


type: ci
user: drh
comment:
  Copy the cache_spill setting from the main database over to the
  vacuum_db transient database when running a VACUUM.


type: ci
user: dan
comment:
  Ensure that the sqlite3_scrub_backup() extension creates a backup database at least as large as indicated by the database header, even if the last page of the input database is a free-list leaf.


type: ci
user: drh
comment:
  Enhance the query planner cost estimation for index scans to take into account
  WHERE clause terms that can be computed using only the index and that do not
  require looking up rows in the original table.  This fixes an obscure
  performance regression that arose when the ORDER BY LIMIT optimization was
  added by check-in [bf46179d44843].


type: ci
user: drh
comment:
  When estimating the cost of an index scan, factor in the cost savings of
  being able to use the index to evaluate some WHERE clause terms without
  having to do a table lookup.


type: ci
user: dan
comment:
  Fix further issues with multi-column IN(...) operators. Also some error handling cases surrounding row values.


type: ci
user: mistachkin
comment: Make sure the SQLITE_TCLAPI macro is always defined.
ecomment:
  Make sure the SQLITE_TCLAPI macro is always defined and use it for all callback functions that must interface with the Tcl C API.


type: ci
user: drh
comment:
  Disable the authorizer callback when reparsing the schema.  This avoids
  undesirable authorization failures following an ALTER TABLE.


type: ci
user: mistachkin
comment:
  Fixup reserved function pointers in the Tcl stubs table and several macros containing Tcl function pointers.


type: ci
user: mistachkin
comment:
  All SQLite API functions and callbacks in the 'sqlite3ext.h' header file must be decorated with the calling convention.


type: ci
user: drh
comment:
  A new version of the slow mutex log that uses gettimeofday() instead of
  trying to access the hardware timer.


type: ci
user: mistachkin
comment:
  Make the extension auto-loading mechanism work with the __stdcall calling convention.  Also, fix a couple Tcl command calling conventions missed in the previous check-in.


type: ci
user: dan
comment:
  Change the way "(a, b) = (SELECT *)" expressions are handled in where.c if there is an index on one of the columns only.


type: ci
user: mistachkin
comment:
  Add macro necessary for building several extensions when the __stdcall calling convention is not in use.


type: g
user: drh
comment:
  Edit [3ea567c4b07b2a7a|3ea567c4b0]: Move to branch [/timeline?r=callbackConv&nd&dp=3ea567c4b07b2a7a&unhide | callbackConv].


type: ci
user: drh
comment:
  Undo some unhelpful changes to skip-scan scoring that were added
  by the check-in [9e2b2681] that improved scoring of index scan.


type: ci
user: mistachkin
comment:
  For test 'shell1-5.0', disable round-trip testing for characters in the range 0xE0 to 0xEF on Linux, due to intermittent shell portability issues.


type: ci
user: dan
comment:
  Fix some issues with vector range constraints and the column cache. Also vector range constraints and rowid columns.


type: ci
user: drh
comment:
  Avoid making unnecessary changes to the signatures of the
  sqlite3_auto_extension() and sqlite3_cancel_auto_extension() interfaces.


type: ci
user: drh
comment:
  Update releasetest.tcl to automatically do one round of tests with
  USE_STDCALL=1 when running on Windows with MSVC.


type: ci
user: dan
comment:
  Add an extra compiler switch to loadext.test to ensure that it picks up the correct version of sqlite3.h when building a test extension.


type: ci
user: drh
comment:
  Update the wordcount.c test program and add a new script "time-wordcount.sh"
  for additional performance testing.


type: ci
user: dan
comment:
  Fix a problem introduced by [77948b5eceab92a7] causing duplicate calls to be made to the xSavepoint() method of virtual tables under some circumstances.


type: ci
user: dan
comment:
  Add tests and fixes for vector operations that use sub-queries with different combinations of LIMIT, OFFSET and ORDER BY clauses.


type: ci
user: drh
comment:
  Make sure the ORDER BY LIMIT optimization is not applied if the inner-most
  loop can only have a single iteration and is hence not really a loop.


type: ci
user: dan
comment:
  Fix a problem with estimating the number of rows visited by a query that uses a multi-column IN(SELECT...) constraint.


type: ci
user: drh
ecomment:
  Add an experimental SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION compile-time option.
  This changes has been merged into main branches via cherry-pick.

comment:
  Add an experimental SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION compile-time option.
  When enabled, the "unknown function" error is suppressed for EXPLAIN and
  a no-op function named "unknown" is substituted.  This facilitiates using
  the command-line shell to analyze queries from applications that contain
  many application-defined functions that are not normally available to the
  shell.


type: ci
user: drh
comment:
  Add the experimental SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION compile-time option.
  When enabled, the "unknown function" error is suppressed for EXPLAIN and
  a no-op function named "unknown()" is substituted.  This facilitiates using
  the command-line shell to analyze queries from applications that contain
  many application-defined functions that are not normally available to the
  shell.


type: ci
user: drh
comment:
  Revert the SQLITE_APICALL changes for now.  That changes needs further
  research before it is released.  Deferred until 3.15.


type: ci
user: drh
comment:
  Remove an incorrect ALWAYS() statement and add a test case to show when
  the condition is sometimes false.


type: ci
user: dan
comment:
  Undo commit [f250166bb]. It is required to handle IO and other errors that occur within a VACUUM of a Zipvfs database.


type: ci
user: dan
comment:
  Have the TEA package build without SQLITE_OMIT_DEPRECATED. As it uses recently deprecated interfaces sqlite3_profile() and sqlite3_trace().


type: ci
user: dan
comment:
  Add a debug timer to pcache1Truncate(). Change the formatting of some other messages to make them easier to read.


type: ci
user: dan
comment:
  Have fts5 interpret column lists that begin with a "-" character as "match any column except" lists.


type: ci
user: drh
comment:
  Fix SQLITE_DBCONFIG_REQUIRE_TXN so that it allows BEGIN IMMEDAITE and
  BEGIN EXCLUSIVE to run outside of a transaction.


type: ci
user: drh
comment:
  Improved comments on the lempar.c parser template.  Adjust the
  YY_SHIFT_USE_DFLT constant in the Lemon-generated parser tables so as
  to guarantee that it is always out of range of the yy_lookahead[] table.


type: ci
user: drh
comment:
  Rearrange the code inside sqlite3RunParser() routine so that sqlite3Parser()
  is only called from a single place.  This allows sqlite3Parser() to be
  in-lined, which results in a 0.25% overall performance gain.


type: ci
user: drh
comment:
  Enhance sqlite3PcacheTruncate() to run faster in the common case where the
  cutoff is just a few pages less than the page number highwater mark.


type: ci
user: drh
comment:
  Merge version 3.14 plus some subsequent patches (including the page-cache
  performance patch) from trunk.


type: ci
user: dan
comment:
  Add the "modeof=<filename>" URI parameter to os_unix.c - used to specify a file to copy permissions from when a new database is created. Also allow passing NULL as the second parameter to sqlite3rbu_vacuum().


type: ci
user: drh
comment:
  Fix a problem in internal function sqlite3OpenTableAndIndices causing an 
  operand of an unrelated VM instruction to be overwritten. Fix for [ef360601].


type: ci
user: drh
comment:
  Attempt to simplify the logic and generated code for vector comparisons.
  Basic comparison operators are working, but there are many indexing test
  failures still to be worked through.


type: ci
user: drh
comment:
  Fix a bug in destructor processing of Lemon.  That has no impact on the
  SQLite grammar.  The bug was introduced by prior work to optimize the
  Lemon-generated parser used by SQLite.


type: ci
user: dan
comment:
  Bias the fts5 snippet() function to return snippets that look like they start at the start of sentences.


type: g
user: drh
comment:
  Edit [60de159476edbd48|60de159476]: Move to branch [/timeline?r=fts5-snippet-bias&nd&dp=60de159476edbd48&unhide | fts5-snippet-bias].


type: ci
user: drh
comment:
  Rename the Db.zName field to Db.zDbSName to make it more descriptive and to
  distinguish it from all of the other "zName" variables scattered throughout
  the code.


type: ci
user: dan
comment:
  Have the fts5 snippet() function avoid favouring snippets that begin with sentences that do not contain search terms. Add an extra bias in favour of the first sentence in the document.


type: ci
user: drh
comment:
  Since the TK_VECTOR token is not actually used by the parser, make it an
  extra token code added by the tool/addopcodes.tcl script during parser build.


type: g
user: drh
comment:
  Edit [92a22f01343a8984|92a22f0134]: Move to branch [/timeline?r=zDbSName&nd&dp=92a22f01343a8984&unhide | zDbSName].


type: ci
user: drh
comment:
  Add the capability to VACUUM an attached database by specifying the schema
  name as an argument to the VACUUM command.  Since version 2.0, VACUUM has
  accepted an argument which was silently ignored.  Now it has meaning.


type: ci
user: dan
comment:
  Register any built-in fts5 module before loading automatic extensions. This allows automatic extensions to register fts5 tokenizers and auxiliary functions.


type: ci
user: drh
comment:
  Replace the magic number (-2) with its symbol XN_EXPR in the
  exprMightBeIndexed() routine.  No logic changes.


type: ci
user: drh
comment:
  Change the way TK_SELECT_COLUMN is handled so that the subquery is only
  generated once even if part of the vector comparison is used for indexing
  and the other part is now.  This change also is a pathway to vector assignment
  in UPDATE statements.


type: ci
user: dan
comment:
  Fix a segfault that could occur if a query that used a vector comparison contained certain types of syntax errors.


type: ci
user: drh
comment:
  Clarification of code comments in expr.c.  Clean up the implementations
  of sqlite3ExprIsVector() and sqlite3ExprVectorSize() slightly.


type: ci
user: drh
comment:
  The docs promise the in "x BETWEEN y AND z" the x expression is only evaluated
  once.  That is no longer true, and so some tests are failing.  This needs to
  be fixed before merging to trunk.


type: ci
user: drh
comment:
  Reinstate the mechanism in BETWEEN that avoids evaluating the first expression
  more than once, but fix the affinity extractor so that it works with this
  mechanism.  The de-duplication of the first expression still does not work
  for vector expressions, though.


type: ci
user: drh
comment:
  Fix the vector BETWEEN operator so that it only evaluates the left-most
  vector expression once.  Add support for vector comparisons in the CASE
  operator.


type: ci
user: drh
comment:
  Fix the SQLITE_USER_AUTHENTICATION login check so that the SQLITE_AUTH_USER
  error code correctly propagates and is not overwritten by SQLITE_ERROR.


type: ci
user: dan
comment:
  Add a new method to sessions - sqlite3sessions_fullchangeset() - to return a changeset that always contains values for all old.* fields. Update changebatch to use these values to more reliably detect multi-column UNIQUE constraint violations.


type: ci
user: drh
comment:
  Fix a problem with rowvalue UPDATE when the rowvalue is not the left-most and
  the RHS is a multi-column subquery.


type: ci
user: dan
comment:
  Fix a buffer overrun in the code for handling IN(...) operators when the LHS of the operator contains indexed columns or expressions.


type: ci
user: drh
comment:
  Improved extended comments of comparison operators when the SQLITE_STOREP2
  flags is set on P5.  No changes to non-debug code.


type: ci
user: drh
comment:
  Improvements to IN operator code generator comments.  Avoid unnecessary
  Copy operations on the LHS of the IN operator.


type: ci
user: drh
comment:
  Fix the sqlite3_trace_v2() interface so that it goes disabled if either the
  callback or mask arguments are zero, in accordance with the documentation.


type: ci
user: dan
comment:
  Fix a problem in internal function sqlite3OpenTableAndIndices causing an operand of an unrelated VM instruction to be overwritten. Fix for [ef360601].


type: ci
user: drh
comment:
  Fix the extra comments (added with -DSQLITE_ENABLE_EXPLAIN_COMMENTS) so that
  the sense of &lt;, &lt;=, &gt;, and &gt;= tests is correct and so that the
  SQLITE_STOREP2 version is shown correctly.  These changes are already in the
  rowvalue branch but are added here since they are technically unrelated to
  rowvalue.


type: t
user: drh
brief: Fixed ticket [ef36060112a50591|ef36060112].
comment:
  Fixed ticket [ef36060112a50591|ef36060112]: <i>Incorrect affinity transformations on DELETE</i> plus 5 other changes


type: ci
user: drh
comment:
  Fix a post-OOM crash in updateRangeAffinityStr().  Add several ALWAYS()
  macros on unreachable branches.


type: ci
user: drh
comment:
  Enhance sqlite3FindInIndex() so that it is able to make use of the
  primary keys at the end of an index.


type: ci
user: drh
comment:
  Fix the sqlite3FindInIndex() to ensure that it always uses a prefix of
  the index and uses no repeated columns.  Enhanced comments.


type: ci
user: drh
comment:
  Fix the "Synopsis" on the OP_Lt, OP_Le, OP_Gt, and OP_Ge opcodes, which
  has been backwards for time out of mind.


type: ci
user: dan
comment:
  If SQLITE_ENABLE_ZIPVFS is defined, journal_mode=off is configured and a savepoint or statement rollback is attempted, move the pager into the error state to prevent the transaction from being committed. This makes it safe to use journal_mode=off with zipvfs under some conditions.


type: ci
user: dan
comment:
  Have "sqldiff --rbu" ignore rows with NULL values in primary key fields. RBU can't handle such rows and the documentation already says sqldiff ignores them. Because the code now uses "=" instead of "IS" to filter on primary key columns, diffs on virtual tables are faster now too.


type: ci
user: dan
comment:
  Within a backup operation, ensure that a read-transaction is opened on the source database before its page size is read. This ensures the page-size used to write to the backup database is the same as its actual page-size, which is important for ZipVFS databases.


type: g
user: drh
comment:
  Edit [3a9f4750a6e65c16|3a9f4750a6]: Move to branch [/timeline?r=mistake&nd&dp=3a9f4750a6e65c16&unhide | mistake].


type: ci
user: dan
comment:
  Consider the affinity of "b" when using an "a IN (SELECT b ...)" expression with an index on "a". Fix for [199df416].


type: t
user: drh
brief: New ticket [199df4168c7bb752|199df4168c].
comment:
  New ticket [199df4168c7bb752|199df4168c] <i>Different answer with and without index on IN operator with type mismatch</i>.


type: t
user: drh
brief: Ticket [199df4168c7bb752|199df4168c]: 6 changes
comment:
  Ticket [199df4168c7bb752|199df4168c] <i>Different answer with and without index on IN operator with type mismatch</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  Build the generate_series(START,END,STEP) table-valued function into
  fuzzershell, to make it easier to construct compact test cases that contain
  actual data.


type: ci
user: dan
comment:
  Fix a problem causing the affinity of sub-select row-value elements to be ignored in some contextes.


type: ci
user: dan
comment:
  Fix a crash that could occur under certain circumstances if the vectors on either side of a comparison operator were of a different size.


type: ci
user: drh
comment:
  Fix an assert() so that it does C-compiler does not combine an assert()
  conditional with a production code conditional and thereby confuse the mutation
  testing script.


type: ci
user: drh
comment:
  Enhance the sqlite3GetTempRange() and sqlite3ReleaseTempRange() internal
  routines so that they use sqlite3GetTempReg() and sqlite3ReleaseTempReg()
  when nReg==1.


type: ci
user: dan
comment:
  Fix a problem handling (a, b) IN (SELECT ...) expressions when there is an index on just one of "a" or "b".


type: ci
user: drh
comment:
  Fix corer cases of vector IN operators where the RHS is a compound SELECT
  that includes an ORDER BY clause.


type: ci
user: drh
comment:
  Fix the ".read" command in the command-line shell so that it understands
  that the input is not interactive.


type: ci
user: drh
comment:
  The ORDER BY LIMIT optimization is not valid unless the inner-most IN operator
  loop is actually used by the query plan.
  Fix for ticket [0c4df46116e90f92].


type: t
user: drh
brief: Fixed ticket [0c4df46116e90f92|0c4df46116].
comment:
  Fixed ticket [0c4df46116e90f92|0c4df46116]: <i>Incorrect result with ORDER BY DESC and LIMIT</i> plus 5 other changes


type: ci
user: drh
comment:
  Correct affinity computations for a SELECT on the RHS of an IN operator.
  Fix for ticket [199df4168c].


type: ci
user: drh
comment:
  Simplify the affinity handling logic in codeAllEqualityTerms().  Logically
  the same, just a little easier to read and understand.


type: ci
user: dan
comment:
  Fix a problem handling expressions like "(a, b) IN (SELECT ... ORDER BY 1, 2)" when there is an index on "a" but not "b".


type: ci
user: drh
comment:
  Add the ext/misc/memvfs.c extension that implements a VFS for read-only
  database files contained in memory.


type: ci
user: mistachkin
comment:
  Fix the extra comments (added with -DSQLITE_ENABLE_EXPLAIN_COMMENTS) so that the sense of <, <=, >, and >= tests is correct and so that the SQLITE_STOREP2 version is shown correctly.  Cherrypick of [4d43c4698eef4e3d].


type: ci
user: drh
comment:
  Add support for row-value comparisons, including IN operators, and
  row-value updates in the UPDATE statement.


type: ci
user: mistachkin
comment:
  Fix the sqlite3_trace_v2() interface so that it goes disabled if either the callback or mask arguments are zero, in accordance with the documentation.  Cherrypick of [37e6c54b1afc6348].


type: ci
user: drh
comment:
  Issue a warning and prompt the user to continue if the releasetest.tcl script
  is run in a checkout with uncommitted changes.


type: ci
user: mistachkin
comment:
  Remove an extra space before function names in the amalgamation.  Cherrypick of [20f3c7436f6a8a7bab39].


type: ci
user: drh
comment:
  In releasetest.tcl:
  Improved testing for uncommitted changes.  If uncommitted changes are seen,
  exit immediately unless the --force option is used.


type: ci
user: drh
comment:
  In releasetest.tcl, the --srcdir option is automatic, so no need to mention
  it in the --help output.


type: ci
user: dan
comment:
  Have releasetest.tcl delete all test files except for "testfixture", "testfixture.exe", "sqlite3", "sqlite3.exe", "test.log" and "test-out.txt" after each test run is finished. Passing the "--keep" option restores the old behaviour.


type: ci
user: drh
comment:
  Update comments used to generate API documentation to include the new
  "&#91;dateof:3.X.Y&#93;" notation for dates of release where appropriate.
  No changes to code.


type: ci
user: drh
comment:
  Fix the ORDER BY LIMIT optimization backport so that it works when the
  ORDER BY uses the DESC direction.


type: ci
user: drh
comment:
  Remove no-op code (adding an assert() to prove that the code was no-op)
  in the text affinity logic of the bytecode engine comparison operators.


type: ci
user: drh
comment:
  Omit the sqlite3Apis constant object when compiling with
  SQLITE_OMIT_LOAD_EXTENSION, since it is not used.


type: ci
user: drh
comment:
  In the command-line shell, add the --new option to the ".open" command.
  Also, report the current database filename as part of the ".show" command.


type: ci
user: drh
comment:
  In the shell, make ".testcase" and ".check" available in all configurations,
  not just SQLITE_DEBUG.  All testcases to be named.  Show the result of each
  testcase on stdout.


type: t
user: drh
brief: Ticket [0eab1ac7591f511d|0eab1ac759]: 6 changes
comment:
  Ticket [0eab1ac7591f511d|0eab1ac759] <i>O(N*N) behavior where it should be O(N)</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  Replace a faulty assert() with a testcase() to assure the condition is
  tested.

ecomment:
  Replace a faulty assert() with a testcase() to assure the condition is
  tested.  Ticket [0eab1ac7591f].


type: ci
user: dan
comment:
  Fix a problem causing incorrect code to be generated for IN constraints like "a IN (1, 2, 3)" where column "a" is a rowid column with an extra UNIQUE index created on it. Ticket [0eab1ac759].


type: t
user: drh
brief: Fixed ticket [0eab1ac7591f511d|0eab1ac759].
comment:
  Fixed ticket [0eab1ac7591f511d|0eab1ac759]: <i>O(N*N) behavior where it should be O(N)</i> plus 3 other changes


type: ci
user: drh
comment:
  Fix problems with the fileRead() function in the command-line shell.  Fix
  a harmless memory leak in the command-line shell, to make validation testing
  easier.


type: ci
user: drh
comment:
  If no KEY is specified on ATTACH, copy the main schema key if the key size
  is non-zero.  In other words, allow for -1 as a valid key size in support
  of the --textkey to SEE.


type: ci
user: mistachkin
comment:
  Fix typo in the name of a Win32 string conversion routine.  Replace several uses of raw_printf() with utf8_printf() in the shell.


type: ci
user: mistachkin
comment:
  Minor enhancement to the TclKit batch tool: it should download the TclKit EXE alone when the TCLKIT_NOENV environment variable is set.


type: ci
user: drh
comment:
  Revise the implementation of OP_Once so that it is smaller, faster, and uses
  less memory.  This also fixes an obscure bug introduced 3 days ago by check-in
  [5990a1bdb4a073].


type: ci
user: drh
comment:
  Fix a segfault introduced by the row-value enhancement that comes up on
  a skip-scan where the first term of the index is unconstrained and the
  second term is of the form "columm IN (SELECT...)".


type: ci
user: drh
comment:
  Improved implementation of 64-bit signed integer multiply that correctly
  detects overflow (and promotes to floating-point) in some corner cases.
  Fix for ticket [1ec41379c9c1e400]


type: t
user: drh
brief: New ticket [1ec41379c9c1e400|1ec41379c9].
comment:
  New ticket [1ec41379c9c1e400|1ec41379c9] <i>Integer multiple fails to overflow to floating-point</i>.


type: t
user: drh
brief: Fixed ticket [1ec41379c9c1e400|1ec41379c9].
comment:
  Fixed ticket [1ec41379c9c1e400|1ec41379c9]: <i>Integer multiple fails to overflow to floating-point</i> plus 5 other changes


type: ci
user: dan
comment:
  Do not run test "delete_db.test" with either the journaltest or inmemoryjournal permutations. Ensure that the multiplexor tests in delete_db.test are performed in non-autovacuum mode.


type: ci
user: dan
comment:
  Fix a problem with the fts5 auxiliary function API causing a crash when a query contained a phrase of zero terms.


type: ci
user: drh
comment:
  Fix speedtest1.c so that it works with SQLITE_OMIT_DEPRECATED.
  Add the --lean and --cachesize options to speed-check.sh.


type: ci
user: drh
comment:
  Remove the internal sqlite3CodeOnce() interface, replacing it with a
  direct call to sqlite3VdbeAddOp0(v,OP_Once).  Slightly smaller and faster.


type: ci
user: drh
comment:
  Fix a potential null-pointer dereference and crash in the case where one
  thread is calling sqlite3_column_text() and another thread is calling
  sqlite3_step() on the same prepared statement at the same instant.


type: ci
user: drh
comment:
  When the [https://www.sqlite.org/queryplanner.html#partialsort|block sorting optimization]
  is used in a scalar subquery, be sure to exit the loop as soon as the first
  valid output row is received.  Fix for ticket [cb3aa0641d9a4] backported
  to the 3.8.9 branch.


type: ci
user: drh
comment:
  Use sqlite3ExprAlloc() instead of sqlite3PExpr() for leaf nodes in the
  expression tree, where appropriate.  This is both smaller and faster.


type: ci
user: drh
comment:
  Add the EP_Leaf flag bit to the Expr.flags field to indicate Expr
  nodes that do not have substructure.  Use that bit to avoid unnecessary
  recursion.


type: ci
user: drh
comment:
  Omit the LikeOp object from the parser.  Change more sqlite3PExpr() calls into
  sqlite3ExprAlloc() calls.


type: ci
user: dan
comment:
  When flattening a query of the form "SELECT * FROM (SELECT * FROM tbl WHERE x=?) WHERE y=?", ensure that the final WHERE clause is "x=? AND y=?" instead of "y=? AND x=?". Although it is still not guaranteed, this makes the order in which WHERE clause terms are processed comport more closely to users expectations.


type: ci
user: drh
comment:
  Remove the peep-hole optimization of removing OP_Close opcodes that come
  before OP_Halt, as the extra work of removing those opcodes uses more cycles
  than just running them.


type: ci
user: drh
comment:
  Fix the Parse.aColCache column cache so that all of the valid entries are 
  in the first Parse.nColCache slots.


type: ci
user: drh
comment:
  Avoid initializing the column-cache section of the Parse object, since entries
  in the cache will be initialized as they are used, and avoiding the initial
  memset() saves many CPU cycles.


type: ci
user: drh
comment:
  Make sure deleting an unused prepared statement does not reference
  uninitialized fields in the structure.


type: ci
user: drh
comment:
  Allocate Parse objects off of the stack where appropriate for a substantial
  performance increase and a size reduction.


type: ci
user: drh
comment:
  Make sure the sqlite_sequence table is updated when an optional xfer
  optimization is used.  Fix for ticket [7b3328086a5c1].


type: t
user: drh
brief: New ticket [7b3328086a5c116c|7b3328086a].
comment:
  New ticket [7b3328086a5c116c|7b3328086a] <i>AUTOINCREMENT problem with the INSERT xfer optimization</i>.


type: t
user: drh
brief: Fixed ticket [7b3328086a5c116c|7b3328086a].
comment:
  Fixed ticket [7b3328086a5c116c|7b3328086a]: <i>AUTOINCREMENT problem with the INSERT xfer optimization</i> plus 5 other changes


type: ci
user: drh
comment:
  Avoid unnecessary strlen() calls in sqlite3ExprAssignVarNumber() by passing in
  the token length from the parser.


type: ci
user: drh
comment:
  Do not set MEM_Blob unless the appropriate fields of the Mem object have
  been correctly initialized.


type: ci
user: dan
comment:
  Update test/trace3.test to account for the fact that casting a pointer to a 64-bit signed integer might produce a negative value.


type: ci
user: drh
comment:
  Make sure indexes on expressions skip over initial NULL values in the
  index.  Fix for ticket [4baa464912129477f3c9]


type: ci
user: dan
comment:
  When handling ORDER BY expressions, do not assume all values of an indexed expressions are distinct. Fix for [4766f444].


type: t
user: drh
brief: Fixed ticket [4baa464912129477|4baa464912].
comment:
  Fixed ticket [4baa464912129477|4baa464912]: <i>NULL handling for indexes on expressions</i> plus 5 other changes


type: t
user: dan
brief: Closed ticket [4766f444868a45c6|4766f44486].
comment:
  Closed ticket [4766f444868a45c6|4766f44486]: <i>ORDER BY handling with indexes on expressions</i> plus 6 other changes


type: ci
user: dan
comment:
  Fix a problem affecting queries that match the pattern (... WHERE ipk IN (....) ORDER BY ? LIMIT ?). Fix for [96c1454c].


type: t
user: drh
brief: New ticket [96c1454cbfd9509a|96c1454cbf].
comment:
  New ticket [96c1454cbfd9509a|96c1454cbf] <i>Incorrect result with ORDER BY DESC and LIMIT (again)</i>.


type: ci
user: drh
comment:
  Add to sqlite3_analyzer command-line options --version and --tclsh, and also
  the undocumented --debug option.


type: t
user: drh
brief: Fixed ticket [96c1454cbfd9509a|96c1454cbf].
comment:
  Fixed ticket [96c1454cbfd9509a|96c1454cbf]: <i>Incorrect result with ORDER BY DESC and LIMIT (again)</i> plus 5 other changes


type: ci
user: drh
comment:
  Avoid reading the -1-th element of an array in the query planner.  Fix to a
  bug introduced by check-in [8e2b25f9b8a7] from earlier today.  Curiously,
  the problem only appeared on 32-bit systems.


type: ci
user: drh
comment:
  Take care to avoid integer overflow when doing the initial page cache
  allocation with an excessively large cache_size setting.


type: ci
user: dan
comment:
  If SQLITE_DIRECT_OVERFLOW_READ is defined at compile time, add "DIRECT_OVERFLOW_READ" to the list of symbols returned by "PRAGMA compile_options".


type: ci
user: drh
comment:
  Add the ability for the PRAGMA statement to accept multiple arguments.
  Currently all arguments other than the first are ignored.


type: ci
user: dan
comment:
  Avoid using the "direct overflow read" optimization to read large blobs if the
  pager layer has a wal file open - even if the database header indicates that
  the db is not a wal database.


type: ci
user: drh
comment:
  In the sessions module, avoid recording a change if an UPDATE statement
  overwrites a column with REAL affinity containing an integer value with 
  the same value.


type: ci
user: dan
comment:
  In the sessions module, avoid recording a change if an UPDATE statement
  overwrites a column with REAL affinity containing an integer value with the same value.


type: ci
user: drh
comment:
  Performance optimization in moveToRoot(). Avoid repeated validity checking of
  the root page on each call. Once is enough.


type: ci
user: drh
comment:
  Merge recent trunk changes, and especially the PRAGMA index_info enhancement
  which is needed on this branch.


type: ci
user: drh
comment:
  Enhance the "PRAGMA index_info" and "PRAGMA index_xinfo" statements so that
  they work on WITHOUT ROWID tables and provide information about the underlying
  index btree that implements the WITHOUT ROWID table.


type: ci
user: dan
comment:
  Add extra tests to check the result of including a %00 escape in a URI
  when ENABLE_URI_00_ERROR is defined.


type: ci
user: dan
comment:
  Add compile time option SQLITE_ENABLE_URI_00_ERROR. If defined, any "%00"
  escape found in a URI is treated as an error.


type: ci
user: dan
comment:
  If all branches of an OR optimize scan that is the rhs of a LEFT JOIN use the
  same index, set the index cursor to return NULL values if there are no matches
  for a row on the lhs.


type: t
user: dan
brief: New ticket [34a579141b2c5acb|34a579141b].
comment:
  New ticket [34a579141b2c5acb|34a579141b] <i>Incorrect results with OR terms in the ON clause of a LEFT JOIN</i>.


type: ci
user: dan
ecomment:
  If all branches of an OR optimize scan that is the rhs of a LEFT JOIN use the
  same index, set the index cursor to return NULL values if there are no matches
  for a row on the lhs.  Fix for ticket [34a579141b2c5ac]

comment:
  If all branches of an OR optimize scan that is the rhs of a LEFT JOIN use the
  same index, set the index cursor to return NULL values if there are no matches
  for a row on the lhs.


type: t
user: drh
brief: Ticket [34a579141b2c5acb|34a579141b]: 6 changes
comment:
  Ticket [34a579141b2c5acb|34a579141b] <i>Incorrect results with OR terms in the ON clause of a LEFT JOIN</i> status still Open with 6 other changes


type: t
user: dan
brief: Closed ticket [34a579141b2c5acb|34a579141b].
comment:
  Closed ticket [34a579141b2c5acb|34a579141b]: <i>Incorrect results with OR terms in the ON clause of a LEFT JOIN</i> plus 3 other changes


type: ci
user: drh
comment:
  Make sure left-join markings are transferred to the virtual scalar
  subexpressions when decomposing a vector comparison in the ON clause of
  a LEFT JOIN.
  Fix for ticket [fef4bb4bd9185ec8f].


type: t
user: drh
brief: New ticket [fef4bb4bd9185ec8|fef4bb4bd9].
comment:
  New ticket [fef4bb4bd9185ec8|fef4bb4bd9] <i>Incorrect result from a LEFT JOIN with a row-value constraint</i>.


type: t
user: drh
brief: Fixed ticket [fef4bb4bd9185ec8|fef4bb4bd9].
comment:
  Fixed ticket [fef4bb4bd9185ec8|fef4bb4bd9]: <i>Incorrect result from a LEFT JOIN with a row-value constraint</i> plus 5 other changes


type: ci
user: dan
comment:
  Add the SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE sqlite3_dbconfig() option - for
  disabling SQLite's default checkpoint-on-close behaviour.


type: ci
user: dan
comment:
  Fix an issue that was causing the new database image to be assembled entirely
  in heap memory when VACUUMing a database, even if it should use a temp file.
  This could cause SQLITE_NOMEM errors when vacuuming very large databases on
  32-bit systems.


type: ci
user: drh
comment:
  Fix an possible NULL pointer deference in the command-line shell that can
  occur when using imposter mode.


type: ci
user: drh
comment:
  In the command-line shell, fix the .imposter command so that it works
  with indexes on expressions.  Fix the ".mode quote" mode so that it works
  with ".headers ON".


type: ci
user: drh
comment:
  If all branches of an OR optimize scan that is the rhs of a LEFT JOIN use the
  same index, set the index cursor to return NULL values if there are no matches
  for a row on the lhs.  Fix for ticket [34a579141b2c5ac]


type: ci
user: dan
comment:
  Fix a test case in vacuum5.test so that it works with -DSQLITE_TEMP_STORE=2
  and -DSQLITE_ENABLE_ATOMIC_WRITE..


type: ci
user: drh
comment:
  Take care not to try to generate code for the ATTACH and DETACH commands
  if there were syntax errors during parsing.
  Fix for ticket [2f1b168ab4d4844]


type: ci
user: dan
comment:
  When opening the *-shm file for a readonly database, try to open it in read-write mode before falling back to readonly. This is in case some other read/write connection within the same process uses the same file descriptor.


type: ci
user: drh
comment:
  Fix the JSON1 extension so that the JSON validator correctly rejects malformed
  backslash escapes within string literals.


type: ci
user: drh
comment:
  Fix OOM error reporting in the json1_group_object() SQL function.  Remove
  an unreachable branch from the JSON1 logic.


type: ci
user: drh
comment:
  Tweaks to speedtest1.c to better reflect mobile usage based on findings in
  the "Pocket Data" paper by Kennedy, Ajay, Challen, and Ziarek.


type: ci
user: drh
comment:
  Remove the "experimental" marking from the sqlite3_preupdate interfaces.
  But be sure all the interface definitions are within 
  documentation.


type: ci
user: dan
comment:
  Avoid storing redundant fields in sorter records when the sort-key and data have
  fields in common (as in "SELECT a FROM t1 ORDER BY 1").


type: ci
user: drh
comment:
  Comment changes that attempt to better explain the behavior of the "seekResult"
  field on VdbeCursor objects and the seekResult parameter to
  sqlite3BtreeInsert() and the OPFLAG_USESEEKRESULT flag on insert opcodes.
  No changes to code.


type: ci
user: drh
comment:
  When doing a REPLACE on a WITHOUT ROWID table with no secondary indexes,
  bypass the OP_NoConflict/OP_Delete sequence and directly overwrite any
  preexisting row, for a performance increase.


type: ci
user: drh
comment:
  Break up the sqlite3BtreeMovetoUnpacked() routine into an eponymous routine
  and sqlite3BtreeMovetoIntkey().  Each routine specializes in a single kind of
  btree.


type: ci
user: drh
comment:
  Add the --raw option to the ".read" dot-command of the command-line shell,
  to cause the named file to be read and sent directly into sqlite3_exec()
  without any interpretation.


type: t
user: drh
brief: Fixed ticket [2f1b168ab4d48442|2f1b168ab4].
comment:
  Fixed ticket [2f1b168ab4d48442|2f1b168ab4]: <i>Crash following syntax error on DETACH</i> plus 5 other changes


type: ci
user: drh
comment:
  Back out the "--raw" option on ".read" in the command-line shell.  Instead,
  fix the command-line shell so that if EOF is reached without seeing a
  final semicolon, it goes ahead and passes the accumulated SQL text to
  SQLite.


type: ci
user: drh
comment:
  Add the test/ossfuzz.c interface adaptor for OSS-FUZZ.  Make previsions for
  testing the adaptor using fuzzcheck.c.


type: ci
user: dan
comment:
  Fix a problem with switching from wal to rollback mode when
  SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE is configured.


type: ci
user: drh
comment:
  Remove obselete and unused logic in the update code generator.
  Fix the sqlite3BtreeMovetoUnpacked() routine so that it remembers the
  rowid of the row that it landed on.


type: ci
user: drh
ecomment:
  Enhance the OP_IdxInsert opcode so that it can used unpacked key values if
  they are available.  Update the code generator to take advantage of this
  new capability.  The speedtest1.c test is about 2.6% faster as a result.
  Later:  This also introduced bug [30027b613b].  Bummer.

comment:
  Enhance the OP_IdxInsert opcode so that it can used unpacked key values if
  they are available.  Update the code generator to take advantage of this
  new capability.  The speedtest1.c test is about 2.6% faster as a result.


type: ci
user: dan
comment:
  Fix an fts5 problem causing a crash in phrase queries where the first token of
  the phrase matches one or more rows but some other token within the phrase
  matches zero.


type: ci
user: drh
comment:
  Avoid unnecessary calls to sqlite3ExprCacheAffinityChange() when generating
  OP_MakeRecord opcodes that do not change any register affinities.


type: ci
user: drh
comment:
  Fix the ossfuzz.c test module so that it does not segfault after a
  "PRAGMA empty_result_callbacks=1;".  Add the ossshell.c program for
  simple command-line testing of ossfuzz.c.


type: ci
user: drh
comment:
  Enhance fuzzcheck.c to accept database-specific configuration parameters in
  the option CONFIG table.  Add the fuzzdata5.db fuzz database containing
  test cases from OSS-FUZZ.


type: ci
user: drh
comment:
  Make the VACUUM command about 9% faster by avoiding unnecessary calls
  to sqlite3BtreeMovetoUnpacked() while copying rowid tables.


type: ci
user: drh
comment:
  Change the OP_Last opcode so that it is a no-op if the cursor is already
  pointing at the end of the b-tree.


type: ci
user: drh
comment:
  Disable the query flattener optimization for SELECT statements that are on
  the RHS of vector IN operators.  This is a hack that fixes the bug
  described in ticket [da7841375186386c]. A better solution that does not
  disable the query flattener is needed, but this will server for the time
  being.


type: t
user: drh
brief: New ticket [da7841375186386c|da78413751].
comment:
  New ticket [da7841375186386c|da78413751] <i>Query flattening causes bad byte-code for a vector IN operator</i>.


type: ci
user: drh
comment:
  Change the --enable-debug option on configure so that it enables the
  ".selecttrace" and ".wheretrace" commands in the command-line shell.


type: ci
user: drh
comment:
  Update test case for opening SHM files read/write on a read-only connection
  so that they are only attempted on Darwin with SQLITE_ENABLE_PRESIST_WAL.


type: ci
user: dan
comment:
  Require that the database handle be in autocommit mode for
  sqlite3_snapshot_get() to succeed. This is because it may open a read
  transaction on the database file.


type: ci
user: dan
comment:
  Fix a bug in sqlite3_snapshot_recover() that could cause subsequent read
  transactions to use out-of-data cache entries.


type: ci
user: dan
comment:
  Remove the requirement to open the wal file before sqlite3_snapshot_recover()
  is called. Also add some comments to new functions.


type: ci
user: drh
comment:
  Enhance speedtest1.c so that is works on older versions of SQLite, before
  the introduction of the group_concat() aggregate function.


type: ci
user: drh
comment:
  Remove unnecessary OP_Close opcodes at the end of INSERT operations, resulting
  in smaller and faster code.


type: ci
user: dan
comment:
  Code constant vector components outside of any loops in CASE and BETWEEN
  expressions. Fix for [1a684656].


type: t
user: drh
brief: New ticket [1a68465637e1b014|1a68465637].
comment:
  New ticket [1a68465637e1b014|1a68465637] <i>Assertion fault on a conditional row-value expression</i>.


type: ci
user: drh
comment:
  Fix the error message for invalid ".mode" arguments in the command-line shell,
  so that it mentions the new "quote" mode.


type: ci
user: dan
comment:
  Fix a test case in vacuum5.test so that it works with the in-memory journal
  permutation. Cherrypick of [81c86401].


type: t
user: drh
brief: Fixed ticket [199df4168c7bb752|199df4168c].
comment:
  Fixed ticket [199df4168c7bb752|199df4168c]: <i>Different answer with and without index on IN operator with type mismatch</i> plus 3 other changes


type: t
user: drh
brief: Fixed ticket [da7841375186386c|da78413751].
comment:
  Fixed ticket [da7841375186386c|da78413751]: <i>Query flattening causes bad byte-code for a vector IN operator</i> plus 5 other changes


type: t
user: drh
brief: Fixed ticket [1a68465637e1b014|1a68465637].
comment:
  Fixed ticket [1a68465637e1b014|1a68465637]: <i>Assertion fault on a conditional row-value expression</i> plus 5 other changes


type: ci
user: drh
comment:
  Performance improvement in sqlite3StrAccumFinish() for the common case where
  no memory allocation is required.


type: ci
user: drh
comment:
  Refactor the sqlite3BtreeKey() and sqlite3BtreeData() internal interfaces
  into sqlite3BtreePayload() and sqlite3BtreePayloadChecked(), respectively.
  This is a continuation of the optimization started by check-in
  [2d831074cf].  The result is a slightly smaller and faster binary.


type: ci
user: drh
comment:
  Clarification of the behavior of sqlite3_stmt_readonly() on BEGIN IMMEDIATE
  statements.  The is comment changes only - no changes to code.


type: ci
user: drh
comment:
  Fix broken hyperlinks to the sqlite3_snapshot objection definition
  in the documentation.  No changes to code.


type: ci
user: drh
comment:
  The documentation says that the built-in date-time functions give undefined
  results for dates before 0000-01-01 and after 9999-12-31.  Change the
  actually implementation so that the answer given is really NULL.  This also
  avoids unnecessary hand-wringing over an signed integer overflow that might
  otherwise occur when processing out-of-bound dates.


type: ci
user: drh
comment:
  Further changes to the date/time functions to suppress harmless signed
  integer overflow warnings that could have occurred when doing out-of-range
  date calculations which, according to the docs, give undefined results.


type: ci
user: drh
comment:
  More improvements to boundary cases in the date/time functions, flowing out
  of branch coverage testing.


type: ci
user: drh
comment:
  Change the OP_DecrJumpZero opcode back to its old behavior of always
  decrementing, except top the decrementing when it reaches the largest negative
  number.


type: ci
user: dan
comment:
  Modify the patternCompare() function (used for GLOB, LIKE) to better handle
  patterns containing multiple wildcard characters ("*", "%").


type: ci
user: drh
comment:
  Faster version of patternCompare() that uses new return values rather than
  an extra parameter to communicate wildcard information back up to parent
  searches.


type: ci
user: drh
comment:
  Performance improvement for GLOB and LIKE matching for patterns with two
  or more multi-character wildcards ("*" or "%").


type: ci
user: dan
comment:
  Avoid clearing the EP_FromJoin flag from terms in ON clauses when flattening
  sub-selects. Possible fix for [2df0107b].


type: t
user: drh
brief: New ticket [2df0107bd268f4d7|2df0107bd2].
comment:
  New ticket [2df0107bd268f4d7|2df0107bd2] <i>Incorrect result from LEFT JOIN with a subquery on the LHS</i>.


type: t
user: drh
brief: Ticket [2df0107bd268f4d7|2df0107bd2]: 6 changes
comment:
  Ticket [2df0107bd268f4d7|2df0107bd2] <i>Incorrect result from LEFT JOIN with a subquery on the LHS</i> status still Open with 6 other changes


type: t
user: drh
brief: Fixed ticket [2df0107bd268f4d7|2df0107bd2].
comment:
  Fixed ticket [2df0107bd268f4d7|2df0107bd2]: <i>Incorrect result from LEFT JOIN with a subquery on the LHS</i> plus 3 other changes


type: ci
user: drh
comment:
  In balance_nonroot, try to combine dropCell/insertCell combinations for the
  dividers into a cell overwrites.  This results in a very small (0.05%)
  performance gain which is probably not worth the added complexity.


type: ci
user: drh
comment:
  Simplify the date/time function logic for improved rebustness and also to
  decrease the size of the binary.


type: ci
user: drh
comment:
  In the LEMON parser-generator, fix the stack overflow processing so that it
  correct invokes the destructor on the top-level of the parse stack.


type: ci
user: dan
comment:
  Prevent the flattening or where-term push-down optimizations from obscuring
  misuses of SQL row values that can lead to crashes or assert() failures.


type: ci
user: drh
comment:
  Always honor the sqlite3.dbOptFlags bitmask, regardless of compile-time
  options.  Continuing fix for ticket [da78413751863].


type: ci
user: drh
comment:
  Fix the showstat4 utility program so that is displays strings using
  standard SQL notation (single quotes) rather than C-style notation.


type: ci
user: drh
comment:
  Avoid unnecessary zeroing of fields in the MemPage object that are going
  to be reinitialized before use anyhow.  A smaller and faster binary results.


type: ci
user: drh
comment:
  In sqlite3BtreeInsert() when replacing a re-existing row, try to overwrite
  the cell directly rather than deallocate and reallocate the cell.


type: ci
user: drh
comment:
  When doing the sqlite3BtreeInsert() overwrite optimization, make sure the
  memcpy() does not extend off the end of the page.


type: ci
user: drh
comment:
  Reorder the fields in the VdbeCursor object so that those that need to be
  bulk zeroed on allocation are grouped at the beginning, and the memset()
  only runs over those fields that really need it.


type: ci
user: drh
comment:
  Avoid signed integer overflow when dealing with a LIMIT and OFFSET whose
  sum exceeds the maximum integer value.


type: ci
user: drh
comment:
  Change the order of comparison for divider cell overflow in balance non-root
  to avoid a (harmless) reference to an uninitialized variable.


type: ci
user: drh
comment:
  Add the --mmap option to the speedtest1 program and to the speed-check.sh
  script that is frequently used to run speedtest1.


type: ci
user: drh
comment:
  Make the sqlite3PagerGet() interface into a virtual method, with different
  implementations based on the current state of the pager.  This gives a small
  performance increase by avoiding unnecessary branches inside the various
  methods.


type: ci
user: drh
comment:
  Convert sqlite3PagerGet() into a pointer-dispatched virtual method.  This
  makes it about 25% faster.


type: ci
user: dan
comment:
  Fix a problem causing SQLite to return false "foreign key violation" errors
  when there is a partial (i.e. WHERE constrained) UNIQUE index on the parent
  key columns. This bug did not cause SQLite to allow illegal data to be
  inserted into the database, only to reject legal operations.


type: ci
user: drh
comment:
  In the command-line shell, in the output of the ".dump", ".schema", and
  ".fullschema" commands, convert CREATE TABLE statements that appear to come
  from shadow tables into CREATE TABLE IF NOT EXISTS statements.


type: ci
user: drh
comment:
  Three times faster sqlite3SrcListAppend() in the common case by avoiding the
  call to sqlite3SrcListEnlarge() for the first allocation.


type: ci
user: drh
comment:
  Adjust the unix makefiles so that "make test" runs fuzzcheck prior to even
  attempting to build the other test programs.


type: ci
user: drh
comment:
  Fix the configure makefile to align it with the manual makefile.  Unsaved
  changes from the previous check-in.


type: ci
user: dan
comment:
  Add the experimental ".fkey_missing_indexes" command to the shell tool. To
  identify indexes that should be created on child keys if FK processing is to
  be enabled.


type: ci
user: drh
comment:
  Code to automatically create eponymous virtual tables for read-only pragmas.
  Compiles, but does not yet work.


type: ci
user: drh
comment:
  Do exponential rather than linear expansion of the SrcList.a array when 
  appending new elements, to reduce the number of malloc() calls.


type: ci
user: drh
comment:
  Fix the cost estimation in the BestIndex method of the eponymous pragma
  virtual table implementation.


type: ci
user: drh
comment:
  Merge enhancements from trunk, and especially the ability to use
  temp.sqlite_master as an alias for sqlite_temp_master.


type: ci
user: dan
comment:
  Fix a problem causing the planner to generate sub-optimal plans for some queries
  that use recursive WITH sub-queries with LIMIT clauses.


type: ci
user: drh
comment:
  Clarify the expectations for the behavior of the xFetch method on the
  sqlite3_pcache_methods object for the case when the key is zero.


type: ci
user: drh
comment:
  Fix minor issues with the ".lint" command implemention in the shell and
  the shell help text related to ".lint".


type: ci
user: drh
comment:
  Built-in PRAGMA statements without side-effects can be invoked as
  table-valued functions by prefixing their name with "pragma_".


type: ci
user: drh
comment:
  Enhance fuzzershell.c to read and execute SQL commands in the autoexec table
  of the database under test.  Add the dbfuzz.c test program combining selected
  features of fuzzershell.c and fuzzcheck.c.


type: ci
user: dan
comment:
  Fix a problem in the shell tools readfile() command causing blobs to be 
  truncated at the first embedded 0x00 byte in release builds, or an assert() to
  fail in a debug build.


type: ci
user: drh
comment:
  Change the OP_IfNotZero opcode so that it decrements register P1 by 1 rather
  than the value in P3, and so that it only decrements if originally positive.
  This avoids decrementing the smallest 64-bit signed integer.


type: ci
user: drh
comment:
  Simplifications to the way UnpackedRecord objects are allocated.  Smaller
  and faster code that also fixes a subtle (currently unreachable) bug.


type: ci
user: drh
comment:
  Change the default lookaside configuration from 500 slots of 128 bytes each to 125 slots of 512 bytes each.  This uses the same amount of memory (64,000 bytes) but seems to perform much better in test applications.


type: ci
user: drh
comment:
  Use the VList object to replace Parse.azVar for tracking the mapping between
  SQL parameter names and parameter numbers.  There is a performance
  improvement, though there are still a few hiccups in the current code.


type: ci
user: drh
comment:
  Use the VList object to capture the mapping between SQL parameter names and
  variable numbers in a single memory allocation.


type: ci
user: drh
comment:
  In the command-line shell, improve the formatting to the ".databases"
  command and on the ".tables" command, only show the schema prefix for
  databases past the first ("main") database.


type: ci
user: drh
comment:
  Change the output format of the ".databases" command in the command-line shell
  so that it shows the schema name, a colon, and the corresponding filename.


type: ci
user: drh
comment:
  The schema name "main" is always an acceptable alias for the primary database
  even if the primary database is renamed using SQLITE_DBCONFIG_MAINDBNAME.


type: ci
user: drh
comment:
  Combine the implementations of the ".tables" and ".indexes" commands in the
  command-line shell.  The ".indexes" command now puts the indexes in multiple
  columns, just like ".tables" and shows all indexes in all attached databases.


type: ci
user: drh
comment:
  Enhance the fuzztest utility with the --prng-seed option.  Always reseed the
  PRNG prior to each test.


type: ci
user: drh
comment:
  Remove an incorrect ALWAYS() macro from balance_nonroot(), which could result
  in corrupt databases if it were optimized out.  This ALWAYS was added 
  only two weeks ago (check-in [f9f2e23bbd68a]) and has never appeared in an
  official release.


type: ci
user: drh
comment:
  During fuzz testing with the fuzzcheck utility program, impose a record
  length limit of 100MB and a limit on the length of LIKE strings of 50 bytes.


type: ci
user: drh
comment:
  Avoid a potential (harmless) signed integer overflow in memory usage
  accounting when calling sqlite3_realloc() to reduce the size of an
  allocation.


type: ci
user: drh
comment:
  Disable new test modules nockpt.test and interrupt2.test for incompatible
  permutations.  Add a CORRUPT_DB term to an assert() in vdbe.c.


type: ci
user: drh
comment:
  Adjust a corruption test case to accommodate the sqlite3BtreeInsert()
  optimization of check-in [0b86fbca66].


type: ci
user: drh
comment:
  Allow sqlite3_interrupt() to be invoked on a database connection that is in
  the process of being closed even if SQLITE_ENABLE_API_ARMOR is defined.


type: ci
user: drh
comment:
  Improvements to the kvtest.c utility.  Added the --cache-size option.
  Additional reporting of version and settings at the end of "./kvtest run".


type: ci
user: drh
comment:
  In kvtest.c: do not show the --cache-size setting on fopen() runs.
  Show progress for the (potentially slow) "export" command.


type: ci
user: drh
comment:
  Add more detail to the header command to further explain how to compile and
  use the kvtest.c utility.


type: ci
user: drh
comment:
  Add the kvtest.c test program used to show that it is many times faster to
  read thumbnail and similar BLOBs out of an SQLite database than it is to read
  them as separate files from the filesystem.


type: ci
user: drh
comment:
  In kvtest.c, use stat() instead of fseek()/ftell() to determine the size of
  a BLOB to be read directly from disk.  This makes the pile-of-files database
  more competative against SQLite.


type: ci
user: drh
comment:
  Encode a 64-bit integer literal in date.c as a constant expression so that
  it works on older compilers.  Also fix a harmless compiler warning in vdbe.c.


type: ci
user: drh
comment:
  Disable the pagerAcquireMapPage() routine if memory-mapped I/O is disabled.
  This fixes a harmless compiler warning on OpenBSD.


type: ci
user: drh
comment:
  Try to move OP_Concat operations outside the inner loop.  This turns out
  to make things very slightly slower, at least in speedtest1.c.


type: g
user: drh
comment:
  Edit [5550e815dd943b15|5550e815dd]: Move to branch [/timeline?r=pcache1-zero-page&nd&dp=5550e815dd943b15&unhide | pcache1-zero-page].


type: ci
user: dan
comment:
  Avoid passing NULL pointers to memcmp() or memcpy(), even when the
  "number-of-bytes" argument is passed 0.


type: ci
user: dan
comment:
  Fix a crash that could occur following an OOM in the group_concat() function
  if the second argument is an SQLITE_BLOB value.


type: ci
user: drh
comment:
  Add the --all option to the wordcount test program.
  Fix the speedtest1 test program so that it builds on MSVC and so that
  the --lookaside 0 0 option works.


type: ci
user: drh
comment:
  Detect row-value comparison size mismatches even when the size of one
  operand is obscured by an unexpanded subquery.


type: ci
user: drh
comment:
  Avoid unnecessary calls to sqlite3BtreeEnterAll() and sqlite3BtreeLeaveAll()
  when no btree is using shared-cache.


type: ci
user: drh
comment:
  Change SQLITE_DEFAULT_PCACHE_INITSZ from 100 to 20, which is experimentally
  determined to be slightly faster.


type: ci
user: drh
comment:
  Provide the SQLITE_DEFAULT_LOOKASIDE compile-time option and make it's default
  value be 1200,100 (raised from 512,125 in the latest release).


type: ci
user: drh
comment:
  Improvements to the way vector assignment size checking is done.  Size checks
  when the RHS is a SELECT are deferred until after "*" wildcards are expanded.


type: ci
user: drh
comment:
  Defer size checking on row-value assignments for when the RHS is a SELECT
  until after the "*" wildcards have been expanded.


type: t
user: drh
brief: Fixed ticket [8c9458e703666e1a|8c9458e703].
comment:
  Fixed ticket [8c9458e703666e1a|8c9458e703]: <i>row-value updates in triggers do not work.</i> plus 5 other changes


type: ci
user: drh
comment:
  Make use of the __buildin_OP_overflow() functions from GCC when doing
  64-bit signed integer arithmetic.


type: ci
user: drh
comment:
  Use the GCC built-in __sync_fetch_and_sub() to make the sqlite3StatusDown()
  routine atomic, and thereby avoid some mutexing.


type: ci
user: drh
comment:
  Attempt to factor out constant functions from the interior of table scans,
  since functions can often be expensive to compute.


type: ci
user: drh
comment:
  Factor constant functions out of the inner loop since they are more expensive
  than the extra OP_Copy needed to move the result into place.


type: ci
user: drh
comment:
  Adjust filename creation in test_sqllog.c so that it uses leading zeros on
  counters, so that files appear in order when sorted lexicographically.


type: t
user: dan
brief: New ticket [30027b613b44f77b|30027b613b].
comment:
  New ticket [30027b613b44f77b|30027b613b] <i>DELETE trigger on WITHOUT ROWID table not fired for REPLACE op</i>.


type: ci
user: drh
comment:
  Improved the comment on the block of code the provides the performance
  optimization originally added by check-in [925840cfdb].  The original
  check-in omitted condition 4, which was the cause of bug [30027b613b].


type: t
user: drh
brief: Ticket [30027b613b44f77b|30027b613b]: 6 changes
comment:
  Ticket [30027b613b44f77b|30027b613b] <i>DELETE trigger on WITHOUT ROWID table not fired for REPLACE op</i> status still Open with 6 other changes


type: ci
user: dan
comment:
  Fix some problems with foreign key processing within REPLACE ops on WITHOUT
  ROWID tables with no triggers or auxiliary indexes.


type: ci
user: drh
ecomment:
  Fix problems in trigger and foreign key handling when doing REPLACE on a
  WITHOUT ROWID table that has no secondary indexes.
  Fix for ticket [30027b613b4].

comment:
  Fix problems in trigger and foreign key handling when doing REPLACE on a
  WITHOUT ROWID table that has no secondary indexes.


type: ci
user: drh
comment:
  Ensure that the sqlite3_value_text() interface returns a buffer that is long
  enough to hold the complete string plus the zero terminator even when the
  input is a zeroblob.  Fix for a problem detected by OSS-Fuzz.


type: ci
user: dan
comment:
  Fix handling the case where a sub-query in a FROM clause is itself a UNION
  ALL, and one side of that UNION ALL is a query on a view that includes an
  ORDER BY. Fix for ticket [190c2507].


type: ci
user: drh
comment:
  Fix problems in trigger and foreign key handling when doing REPLACE on a
  WITHOUT ROWID table that has no secondary indexes.
  Fix for ticket [30027b613b4].


type: t
user: drh
brief: Fixed ticket [30027b613b44f77b|30027b613b].
comment:
  Fixed ticket [30027b613b44f77b|30027b613b]: <i>DELETE trigger on WITHOUT ROWID table not fired for REPLACE op</i> plus 4 other changes


type: ci
user: dan
comment:
  Ensure that sqlite3_column_count() returns 0 for the "set" mode of "get/set"
  PRAGMA statements that do not return a value in that case (e.g. page_size,
  cache_size, auto_vacuum).


type: t
user: drh
brief: Fixed ticket [190c2507eeffcd11|190c2507ee].
comment:
  Fixed ticket [190c2507eeffcd11|190c2507ee]: <i>Assertion fault on a query against a view</i> plus 5 other changes


type: ci
user: dan
comment:
  Ensure that "PRAGMA case_sensitive_like" and "PRAGMA shrink_memory" set the
  number of output columns to 0 (as they are statements that return no data).


type: ci
user: drh
comment:
  This hack illustrates how to use the VDBE_PROFILE mechanism to show which
  bytecode operators are using resources other than time.  In this case, the
  number of loops through the binary search code in sqlite3BtreeMovetoUnpacked()
  is measured, for the purpose of helping to identify unnecessary btree searches.


type: ci
user: drh
comment:
  Merge latest changes from trunk, and also move the perf-counter into the
  inner loop of sqlite3BtreeMovetoUnpacked().


type: ci
user: drh
comment:
  Avoid an unnecessary btree seek while deleting an index entry due to a conflict
  on a REPLACE operation.


type: ci
user: drh
comment:
  Critical fix to the previous check-in so that it works when there are
  BEFORE triggers that move the cursor before the OP_Delete has a chance to
  be applied.


type: ci
user: drh
comment:
  Improvements to the iIdxNoSeek optimization of sqlite3GenerateRowDelete()
  so that it is automatically disabled for BEFORE triggers but works in all
  other cases.


type: ci
user: dan
comment:
  Disable the optimization from [8cb8516d] (omit TableLock instructions for
  non-sharable databases) on this branch. This branch uses the TableLock
  instructions to ensure that the db schema is not written from within a BEGIN
  CONCURRENT transaction.


type: ci
user: drh
comment:
  Modify the OP_RowData opcode so that when P3!=0 it is allowed to hold an
  ephemeral copy of the content.  This avoids unnecessary memcpy() operations
  in the xfer-optimization and VACUUM.


type: ci
user: drh
comment:
  Avoid unnecessary calls to the xRoundup() method of the memory allocator when
  the soft heap limit is not set.


type: ci
user: drh
comment:
  Throw an error if the ON clause of a LEFT JOIN references tables to the right
  of the ON clause.  Fix for ticket [25e335f802dd].


type: t
user: drh
brief: New ticket [25e335f802ddc1ac|25e335f802].
comment:
  New ticket [25e335f802ddc1ac|25e335f802] <i>Incorrect result when ON clause refers to table to the right</i>.


type: t
user: drh
brief: Ticket [25e335f802ddc1ac|25e335f802]: 6 changes
comment:
  Ticket [25e335f802ddc1ac|25e335f802] <i>Incorrect result when ON clause refers to table to the right</i> status still Open with 6 other changes


type: t
user: drh
brief: Fixed ticket [25e335f802ddc1ac|25e335f802].
comment:
  Fixed ticket [25e335f802ddc1ac|25e335f802]: <i>Incorrect result when ON clause refers to table to the right</i> plus 4 other changes


type: ci
user: dan
comment:
  Fix a problem preventing UPDATE statements that use a range-scan on the PK
  index of a WITHOUT ROWID table from using a one-pass strategy.


type: ci
user: drh
comment:
  When planning a query using sorting, resolve ties in the solver by selecting
  loop plans with the smaller unsorted cost.


type: ci
user: dan
comment:
  Fix a problem causing the pre-update hook to be passed an incorrect rowid
  value in some single-pass multi-row updates.


type: ci
user: drh
comment:
  Improved detection of cells that extend into the reserved space at the end
  of the page while adjusting overflow page pointers during autovacuum.


type: ci
user: drh
comment:
  An example showing how to improve performance of sqlite3VdbeSerialPut()
  using the GCC intrinsic function __builtin_bswap64().


type: ci
user: drh
comment:
  Remove an unnecessary corruption test from the btree balancer.  If corruption
  is present, it will be found harmlessly by later tests.


type: ci
user: dan
comment:
  Fix a problem preventing resumption of RBU operations after recovering from a
  process or system failure that occurs during the incremental-checkpoint phase.


type: ci
user: mistachkin
comment:
  When determining sector sizes on Windows 7 and Vista, make sure the target file is on the same volume as corresponding root directory.


type: ci
user: mistachkin
comment:
  For Windows 8.x and higher, require the SQLITE_WIN32_WIN8_SECTOR_SIZE compile-time option in order to enable sector size determination.


type: ci
user: drh
comment:
  Back out check-in [0b3174e0b1364c] and replace it with a better fix
  for ticket [91e2e8ba6ff2e2] - a fix that does not cause the problem
  identified by ticket [7ffd1ca1d2ad4ec].


type: ci
user: drh
comment:
  Back out check-in [0b3174e0b1364c] and replace it with a better fix for \ticket [91e2e8ba6ff2e2] - a fix that does not cause the problem identified by 
  ticket [7ffd1ca1d2ad4ec]. Add new test cases for both tickets.


type: ci
user: drh
comment:
  Disable intrinsic functions for Windows using Clang, due to reports of
  linkage errors.  This causes a 0.6% performance reduction.  We will want to
  revisit this change in the future.


type: t
user: drh
brief: Fixed ticket [7ffd1ca1d2ad4ecf|7ffd1ca1d2].
comment:
  Fixed ticket [7ffd1ca1d2ad4ecf|7ffd1ca1d2]: <i>Incorrect affinity when using automatic indexes</i> plus 5 other changes


type: ci
user: dan
comment:
  Fix a problem that could cause a spurious SQLITE_NOMEM error when attempting
  to resume an RBU operation if the previous client failed right after
  completing the incremental checkpoint. Also a "cannot vacuum wal db" error
  that could occur when resuming an RBU vacuum if an error (OOM or IO error)
  occurs during the incremental checkpoint.


type: ci
user: dan
comment:
  Add temporary code to record and report on the set of b-tree pages read and
  written by the current transaction. This is likely still buggy.


type: ci
user: mistachkin
comment:
  Fix handling of initial hidden and/or system files in the opendir() implementation for Windows.  No changes to non-test code.


type: ci
user: drh
comment:
  A proof-of-concept for running sqlite3_blob_open() without using OP_Column
  when operating on a pure key/value table.  This demo does not include any
  corrupt database checking.  Uses about 3% fewer CPU cycles on a key/value
  performance test.


type: ci
user: drh
comment:
  If compiled with SQLITE_INLINE_MEMCPY, all memcpy() calls are replaced with
  in-line code.  With that change, cachegrind shows which memcpy() calls
  are taking the most time.  This is a performance-measurement hack only and
  is not for production use.


type: ci
user: drh
comment:
  Get the "--testset rtree" option working on speedtest1.  Add the --rtree,
  --lookaside, and --clang options to the speed-check.sh script.


type: ci
user: dan
comment:
  Add option "--stats" to test program kvtest. Specifying --stats causes kvtest
  to output information similar to the shell tool option of the same name.


type: ci
user: drh
comment:
  Remove an unnecessary sqlite3_bind_int64() call from sqlite3_blob_open().
  Also other minor refactoring of the sqlite3_blob implementation.


type: ci
user: drh
comment:
  In the kvtest.c test utility, reuse the buffer into which blobs are read,
  rather than reallocating it for each row.  This is a closer match to how
  other test programs work, and thus provides a better comparison.


type: ci
user: drh
comment:
  B-tree optimization:  When seeking on a rowid table that has already been
  positioned, check to see if the new row happens to be the next row on the
  same leaf page.  That is a reasonably common case, and if it is true it
  avoids a full binary search.


type: ci
user: drh
comment:
  Change sqlite3_blob_reopen() to call sqlite3VdbeExec() directly rather than
  going through sqlite3_step().  Performance enhancement.


type: ci
user: drh
comment:
  A better implementation of the moveto-neighbor optimization that checks for
  nearby rows on adjacent pages.


type: g
user: drh
comment:
  Edit [8e5cfb2039126da7|8e5cfb2039]: Move to branch [/timeline?r=btree-moveto-neighbor&nd&dp=8e5cfb2039126da7&unhide | btree-moveto-neighbor].


type: ci
user: drh
comment:
  Optimization: Try to avoid unnecessary btree searching when repositioning 
  a cursor to the next row.


type: ci
user: dan
comment:
  Fix a problem preventing FTS5 from correctly passing the FTS5_TOKENIZE_PREFIX
  flag to custom tokenizer implementations.


type: ci
user: dan
comment:
  Add the missing SQLITE_API symbol to test_delete.c (it is not added
  automatically as this file is not part of the amalgamation).


type: ci
user: drh
comment:
  Document the --mmap option in the --help screen for kvtest.  Enhance kvtest so
  that numeric arguments can have suffixes like "K" or "M".  Add kvtest to the
  unix makefiles.


type: t
user: drh
brief: New ticket [e6e962d6b0f06f46|e6e962d6b0].
comment:
  New ticket [e6e962d6b0f06f46|e6e962d6b0] <i>sqlite3_blob_reopen() misbehavior on ALTER-ed table</i>.


type: ci
user: drh
comment:
  Ensure that sqlite3_blob_reopen() correctly handles short rows. 
  Fix for ticket [e6e962d6b0f06f46e].


type: ci
user: drh
comment:
  Ensure that sqlite3_blob_reopen() correctly handles short rows.
  Proposed fix for ticket [e6e962d6b0f06f46e].  Further testing needed.


type: ci
user: drh
comment:
  Experimental enhancement to automatically trim NULL values from the end of
  records, for a reduced disk footprint.  This change also involves increasing
  the P5 operand from 8 to 16 bits.


type: ci
user: drh
ecomment:
  Trim NULL values off the end of records when the SQLITE_ENABLE_NULL_TRIM
  compile-time option is used.  Increase the size of the P5 operand to 16 bits.
  Fix a problem with short records in the sessions extension.

comment:
  Trim NULL values off the end of records when the SQLITE_ENABLE_TRIM_NULLS
  compile-time option is used.  Increase the size of the P5 operand to 16 bits.
  Fix a problem with short records in the sessions extension.


