# event

type: ci
user: dan
comment:
  Add "PRAGMA foreign_keys=OFF;" to the start of the script output by ".recover", just as is done for ".dump".


type: t
user: drh
brief: Fixed ticket [61c853857f40da49|61c853857f].
comment:
  Fixed ticket [61c853857f40da49|61c853857f]: <i>Row is not fetched in SELECT from VIEW</i> plus 6 other changes


type: t
user: drh
brief: Fixed ticket [9f66f887d4a25b05|9f66f887d4].
comment:
  Fixed ticket [9f66f887d4a25b05|9f66f887d4]: <i>EXPLAIN PRAGMA executes PRAGMA</i> plus 8 other changes


type: ci
user: drh
comment:
  Enhance the ".recover" output in the shell to use double-quotes around table
  and column identifiers.


type: ci
user: drh
comment:
  Improved reuse of file descriptors for which close() is delayed to prevent
  clearly of posix advisory locks.


type: t
user: mrigger
brief: New ticket [0a5e2c1dcbefeae5|0a5e2c1dcb].
comment:
  New ticket [0a5e2c1dcbefeae5|0a5e2c1dcb] <i>Unexpected affinity conversion for view column in IN operator</i>.


type: ci
user: dan
comment:
  Ensure that when the col in an operator like "val IN(col)" is a column of a view, its affinity is not used to coerce val. Fix for [0a5e2c1d].


type: ci
user: drh
comment:
  Do not make SQLITE_READ authorizer calls for tables without names, as all
  such tables will be internal-use-only tables for subqueries and whatnot.


type: t
user: drh
brief: Fixed ticket [0a5e2c1dcbefeae5|0a5e2c1dcb].
comment:
  Fixed ticket [0a5e2c1dcbefeae5|0a5e2c1dcb]: <i>Unexpected affinity conversion for view column in IN operator</i> plus 6 other changes


type: ci
user: dan
comment:
  Add "set TMP=%CD%" to the start of each msvc script output by releasetest_data.tcl. Otherwise, since binaries compiled with SQLITE_TEST all choose the same sequence of pseudo-random numbers, collisions between temp file names cause errors when running multiple tests in parallel.


type: ci
user: dan
comment:
  Remove use of the affinity() function from view.test, as it is only available in SQLITE_DEBUG builds.


type: t
user: mrigger
brief: New ticket [f8a7060ece7252a4|f8a7060ece].
comment:
  New ticket [f8a7060ece7252a4|f8a7060ece] <i>Incorrect result for query that uses MIN() and a CAST on rowid</i>.


type: t
user: drh
brief: Ticket [f8a7060ece7252a4|f8a7060ece]: 7 changes
comment:
  Ticket [f8a7060ece7252a4|f8a7060ece] <i>Incorrect result for query that uses MIN() and a CAST on rowid</i> status still Open with 7 other changes


type: ci
user: drh
comment:
  Remove support for STAT3.  The sqlite_stat3 tables are ignored, if they
  exist.  STAT4 continues to work as it always has, and as it is a superset of
  STAT3 is the recommended replacement.


type: t
user: dan
brief: Ticket [f8a7060ece7252a4|f8a7060ece]: 3 changes
comment:
  Ticket [f8a7060ece7252a4|f8a7060ece] <i>Incorrect result for query that uses MIN() and a CAST on rowid</i> status still Open with 3 other changes


type: ci
user: dan
comment:
  Fix spurious "no such table: x1" errors in shared-schema mode that could occur
  when a query that is a join across two or more databases needs to call the xConnect() methods of a virtual table.


type: ci
user: dan
comment:
  In shared-schema mode, handle the case where a connection has created a virtual-table object, but is later assigned a different shared-schema object for which the virtual-table schema has not yet been initialized.


type: ci
user: drh
comment:
  Remove support for SQLITE_ENABLE_STAT3.  The sqlite_stat3 table is now ignored,
  if it exists.  Run ANALYZE using STAT4 to get the equivalent functionality,
  which presumably everybody has been doing for a long time now.


type: ci
user: drh
comment:
  Fix a case of the Expr.affinity to Expr.affExpr refactor that was missed
  in the [a29f2a7d07beff64] check-in.


type: ci
user: drh
comment:
  Remove two incorrect assert() statements from the logic used to derive
  column names and types from subqueries.


type: t
user: mrigger
brief: New ticket [9080b6227fabb466|9080b6227f].
comment:
  New ticket [9080b6227fabb466|9080b6227f] <i>Constant expression in partial index results in row not being fetched</i>.


type: ci
user: drh
ecomment:
  Fix the sqliteExprImpliesExpr() routine so that it recognizes that
  "(NULL IS FALSE) IS FALSE" doe not imply "NULL NOT NULL".
  Ticket [9080b6227fabb466]

comment:
  Fix the sqliteExprImpliesExpr() routine so that it recognizes that
  "(NULL IS FALSE) IS FALSE" doe not implie "NULL NOT NULL".
  Ticket [9080b6227fabb466]


type: ci
user: drh
comment:
  Validate the type, name, and tbl_name fields of the sqlite_master table when
  loading the schema, unless writable_schema is engaged.


type: t
user: drh
brief: Fixed ticket [9080b6227fabb466|9080b6227f].
comment:
  Fixed ticket [9080b6227fabb466|9080b6227f]: <i>Constant expression in partial index results in row not being fetched</i> plus 6 other changes


type: ci
user: drh
comment:
  Add a TESTCTRL that disables the verification of type, name, and tbl_name
  in the sqlite_master table.


type: ci
user: dan
comment:
  Experimental implementation of NULLS FIRST/LAST. This branch still has problems - the most significant of which is that ORDER BY clauses with a non-default NULLS FIRST/LAST qualifier can never use an index.


type: ci
user: drh
comment:
  For the ".testctrl prng_seed" command in the CLI, if the argument is "random"
  then select a random integer seed and print the seed value on stdout.


type: ci
user: dan
comment:
  Add support for using indexes for some ORDER BY clauses that use non-default NULL handling. Still some problems on this branch.


type: ci
user: drh
comment:
  Make sure the btree cursor overflow cache is cleared when overwriting
  a cell in sqlite3BtreeInsert().  Ticket [3cf9bb227e9a5d32]


type: t
user: drh
brief: Fixed ticket [3cf9bb227e9a5d32|3cf9bb227e].
comment:
  Fixed ticket [3cf9bb227e9a5d32|3cf9bb227e]: <i>Database corruption caused by REPLACE</i> plus 5 other changes


type: ci
user: drh
comment:
  Always check for reads past the end of the file on the in-memory journal
  driver.  This used to be an assert().


type: ci
user: drh
comment:
  Early detection out-of-bounds page numbers on the direct-overflow-read
  optimization gives consistent error messages regardless of whether or not
  the optimization is enabled.


type: t
user: dan
brief: Closed ticket [17db54d7440281f2|17db54d744].
comment:
  Closed ticket [17db54d7440281f2|17db54d744]: <i>Unexpected error in DELETE with existing trigger</i> plus 6 other changes


type: ci
user: drh
comment:
  Avoid downgrading SQLITE_CORRUPT errors detected by the schema parser into
  SQLITE_NOMEM or SQLITE_ERROR errors.


type: ci
user: dan
comment:
  Ensure that SQLite does not attempt to process incompatible window functions in a single scan. Fix for [256741a1].


type: t
user: dan
brief: New ticket [256741a16be70df9|256741a16b].
comment:
  New ticket [256741a16be70df9|256741a16b] <i>null pointer dereference caused by window functions in result-set of EXISTS(SELECT ...)</i>.


type: ci
user: drh
comment:
  Ensure that the optional "sz=N" parameter that can be manually added to the
  end of an sqlite_stat1 entry does not have an N value that is too small.
  Ticket [e4598ecbdd18bd82]


type: t
user: dan
brief: Ticket [256741a16be70df9|256741a16b]: 6 changes
comment:
  Ticket [256741a16be70df9|256741a16b] <i>null pointer dereference caused by window functions in result-set of EXISTS(SELECT ...)</i> status still Open with 6 other changes


type: ci
user: drh
comment:
  Provide the SQLITE_DIRECTONLY flag for app-defined functions that prohibits
  the use of those functions within triggers or views.


type: t
user: drh
brief: Fixed ticket [e4598ecbdd18bd82|e4598ecbdd].
comment:
  Fixed ticket [e4598ecbdd18bd82|e4598ecbdd]: <i>Division by zero in the query planner.</i> plus 5 other changes


type: t
user: drh
brief: Fixed ticket [256741a16be70df9|256741a16b].
comment:
  Fixed ticket [256741a16be70df9|256741a16b]: <i>null pointer dereference caused by window functions in result-set of EXISTS(SELECT ...)</i> plus 3 other changes


type: ci
user: drh
comment:
  The SQLITE_DIRECTONLY flag, when added to sqlite3_create_function() prevents
  the function from being used inside a trigger or view.


type: ci
user: drh
comment:
  Add the SQLITE_DBCONFIG_ENABLE_VIEW option, together with a "db config"
  command in the TCL interface that can access that option as well as all the
  other sqlite3_db_config() boolean options.


type: ci
user: drh
comment:
  Add the ability to unregister a virtual table module by invoking
  sqlite3_create_module() with a NULL sqlite3_module pointer.


type: ci
user: drh
comment:
  Activate introspection pragmas by default.  The new option
  SQLITE_OMIT_INTROSPECTION_PRAGMAS must be provided to keep them out.


type: ci
user: drh
comment:
  Ensure the functions that appear to be constant are not factored out of
  expression that originate on the right-hand side of a LEFT JOIN.
  Ticket [6710d2f7a13a2997]


type: ci
user: dan
comment:
  When populating an ephemeral b-tree for the RHS of an IN(...) clause, avoid applying an affinity to a value that may be used later on for some other purpose. Fix for [c7a117190].


type: t
user: drh
brief: Fixed ticket [6710d2f7a13a2997|6710d2f7a1].
comment:
  Fixed ticket [6710d2f7a13a2997|6710d2f7a1]: <i>LEFT JOIN fails to fetch row</i> plus 7 other changes


type: t
user: mrigger
brief: New ticket [c7a11719072efe44|c7a1171907].
comment:
  New ticket [c7a11719072efe44|c7a1171907] <i>WHERE clause erroneously influences value of fetched column from view</i>.


type: ci
user: drh
comment:
  Fix the new ability to unregister virtual table modules so that it works
  for the automatic PRAGMA virtual tables.


type: t
user: dan
brief: Closed ticket [c7a11719072efe44|c7a1171907].
comment:
  Closed ticket [c7a11719072efe44|c7a1171907]: <i>WHERE clause erroneously influences value of fetched column from view</i> plus 6 other changes


type: ci
user: dan
comment:
  Retain the affinity of an expression in a WHERE clause when it is transformed to a reference to an index column on the same expression. Fix for [f043b113].


type: t
user: dan
brief: Closed ticket [f043b1130b16826a|f043b1130b].
comment:
  Closed ticket [f043b1130b16826a|f043b1130b]: <i>INDEXED BY results in row not being fetched</i> plus 6 other changes


type: ci
user: dan
comment:
  Fix a segfault that could occur following an OOM while processing a SELECT statement for which one or more of the expressions in the window frame declaration are themselves sub-selects that use window functions.


type: ci
user: dan
comment:
  Fix a broken assert() in the fts3 snippet code that was failing for queries containging more than 64 phrases.


type: ci
user: dan
comment:
  Avoid assuming that "column IS ?", where column is declared UNIQUE, matches only a single row (as "?" might be NULL). Fix for [b8689402].


type: t
user: dan
brief: Closed ticket [b86894020e985dc5|b86894020e].
comment:
  Closed ticket [b86894020e985dc5|b86894020e]: <i>DISTINCT malfunctions for IS NULL</i> plus 6 other changes


type: ci
user: drh
comment:
  Merge fixes from trunk.  Also fix a reference to the KeyInfo.aSortOrder field,
  which should now be KeyInfo.aSortFlags


type: t
user: mrigger
brief: New ticket [d9f584e936c7a8d0|d9f584e936].
comment:
  New ticket [d9f584e936c7a8d0|d9f584e936] <i>Row is not fetched in table with INTEGER PRIMARY KEY</i>.


type: ci
user: drh
comment:
  Fix the OP_SeekGE, OP_SeekGT, OP_SeekLE, and OP_SeekLT opcodes so that they
  preserve the datatype of the value in the register used as the key.
  Ticket [d9f584e936c7a8d0]


type: t
user: drh
brief: Fixed ticket [d9f584e936c7a8d0|d9f584e936].
comment:
  Fixed ticket [d9f584e936c7a8d0|d9f584e936]: <i>Row is not fetched in table with INTEGER PRIMARY KEY</i> plus 7 other changes


type: t
user: mrigger
brief: New ticket [ac184eb571d5e6e0|ac184eb571].
comment:
  New ticket [ac184eb571d5e6e0|ac184eb571] <i>Row with comparison on TEXT UNIQUE column  is not fetched</i>.


type: ci
user: drh
comment:
  Fix the likely(), unlikely(), and likelihood() functions so that they have
  no affinity, just like any other function.
  Ticket [7e07a3dbf5a8cd26]


type: t
user: drh
brief: Fixed ticket [ac184eb571d5e6e0|ac184eb571].
comment:
  Fixed ticket [ac184eb571d5e6e0|ac184eb571]: <i>Row with comparison on TEXT UNIQUE column  is not fetched</i> plus 7 other changes


type: t
user: mrigger
brief: New ticket [7e07a3dbf5a8cd26|7e07a3dbf5].
comment:
  New ticket [7e07a3dbf5a8cd26|7e07a3dbf5] <i>LIKELY() seems to cause unexpected affinity conversion for rowid</i>.


type: t
user: drh
brief: Fixed ticket [7e07a3dbf5a8cd26|7e07a3dbf5].
comment:
  Fixed ticket [7e07a3dbf5a8cd26|7e07a3dbf5]: <i>LIKELY() seems to cause unexpected affinity conversion for rowid</i> plus 6 other changes


type: ci
user: drh
comment:
  Extra VDBE comments in the NULLS LAST logic provide landmarks to help
  understand the EXPLAIN output.


type: ci
user: drh
comment:
  Invert the meaning of the regBignull flag so that it is 1 when doing the
  normal scan and 1 when scanning nulls.  This enables the re-do jump at the
  bottom of the loop to be coded with a single OP_IfNotZero opcode, rather
  than a sequence of OP_If, OP_Integer, OP_Goto.


type: ci
user: drh
comment:
  Simplification of the WHERE loop code generator for NULLS LAST saves a
  few CPU cycles and about a hundred bytes of code space.


type: ci
user: drh
comment:
  Revise the VDBE comments for NULL-scan so that they also work make
  sense when reading a NULLS FIRST plan.


type: ci
user: dan
comment:
  Move some things in parse.y so that TK_COLUMN and TK_AGG_COLUMN are assigned the same values as they are on trunk for a very small speedup.


type: ci
user: drh
comment:
  Attempt to parse the NULLS LAST clause more efficiently.  A few cycles were
  saved, but at the expense of more code.  And there is a bug was introduced
  somewhere.  Not worth continuing down this line.


type: g
user: dan
comment:
  Edit [c88d42a921edd00f|c88d42a921]: Move to branch [/timeline?r=mistake&nd&dp=c88d42a921edd00f&unhide | mistake]. Add propagating "hidden". Mark "Closed".


type: ci
user: drh
comment:
  Improved detection of number of column mismatch for vector assignment
  in UPDATE statements.  Ticket [78acc9d40f0786e8]


type: ci
user: dan
comment:
  Tweak the shell tool ".recover" command so that it can recover rows that consist of a rowid and no fields.


type: t
user: drh
brief: Fixed ticket [78acc9d40f0786e8|78acc9d40f].
comment:
  Fixed ticket [78acc9d40f0786e8|78acc9d40f]: <i>Assertion fault in row-value UPDATE</i> plus 5 other changes


type: t
user: drh
brief: Ticket [78acc9d40f0786e8|78acc9d40f]: 3 changes
comment:
  Ticket [78acc9d40f0786e8|78acc9d40f] <i>Assertion fault in row-value UPDATE</i> status still Fixed with 3 other changes


type: ci
user: dan
comment:
  Add support for "ORDER BY ... NULLS FIRST" and "ORDER BY ... NULLS LAST". Use this to fix ticket [f8a7060e].


type: t
user: dan
brief: New ticket [ac8dd4a32ba4322f|ac8dd4a32b].
comment:
  New ticket [ac8dd4a32ba4322f|ac8dd4a32b] <i>Segfault when dropping a temp trigger on a database that has already been detached</i>.


type: ci
user: drh
comment:
  Omit the "x IN (y)" to "x==y" optimization of check-in [e68b427afbc82e20]
  (and ticket [e39d032577df6942]) as it causes difficult affinity problems
  as demonstrated by ticket [dbaf8a6820be1ece] and the original assertion fault
  is no longer a factor due to countless other changes of the previous 5 years.


type: ci
user: drh
ecomment:
  If a TEMP TRIGGER references an auxiliary schema, and that auxiliary schema
  is detached, move the trigger to reference the TEMP schema before completing
  the detach, so that the trigger does not hold a dangling schema pointer.
  Ticket [ac8dd4a32ba4322f]

comment:
  If a TEMP TRIGGER references an auxiliary schema, and that auxiliary schema
  is detached, move the trigger to reference the TEMP schema before completing
  the detach, so that the trigger does not hold a dangling schema pointer.


type: t
user: drh
brief: Fixed ticket [ac8dd4a32ba4322f|ac8dd4a32b].
comment:
  Fixed ticket [ac8dd4a32ba4322f|ac8dd4a32b]: <i>Segfault when dropping a temp trigger on a database that has already been detached</i> plus 5 other changes


type: t
user: mrigger
brief: New ticket [dbaf8a6820be1ece|dbaf8a6820].
comment:
  New ticket [dbaf8a6820be1ece|dbaf8a6820] <i>Unexpected affinity conversion is performed for the IN operator</i>.


type: ci
user: drh
comment:
  Add ALWAYS() to an always true conditional that results from the
  previous check-in.  Add a test case for ticket [dbaf8a6820be1ece] to
  supplement those already checked into TH3.


type: t
user: drh
brief: Fixed ticket [dbaf8a6820be1ece|dbaf8a6820].
comment:
  Fixed ticket [dbaf8a6820be1ece|dbaf8a6820]: <i>Unexpected affinity conversion is performed for the IN operator</i> plus 6 other changes


type: t
user: dan
brief: Closed ticket [f8a7060ece7252a4|f8a7060ece].
comment:
  Closed ticket [f8a7060ece7252a4|f8a7060ece]: <i>Incorrect result for query that uses MIN() and a CAST on rowid</i> plus 4 other changes


# attachment

src: 413d1b10a5d94b0bdf1f13df5f6e6142a8d25e5c
target: 1c67bd6ce9d7fd366fdc92123af91da0750256b7
filename: control_c.patch
user: anonymous
comment:
  This was bothering me so I thought I'd attempt a patch.  This is my first time working with the SQLite code so please tell me if anything seems wrong.  I'm submitting this as anonymous because I can't figure out how to create an account here.

  The author or authors of this code dedicate any and all copyright interest in this code to the public domain. We make this dedication for the benefit of the public at large and to the detriment of our heirs and successors. We intend this dedication to be an overt act of relinquishment in perpetuity of all present and future rights to this code under copyright law. 


# ticket

tkt_uuid: c25aab7e7ea55c861313076cae9257fc8f3fbdc0
type: Feature_Request
status: Fixed
subsystem: Shell
priority: Low
severity: Important
foundin: 3.6.23.1
private_contact: 09913f947bd60d606711475609f369679c784ca2
resolution: Fixed
title: The ".import" command does not parse CSV correctly.
comment:
  when importing CSV data, in text fields separated by double quotes, commas are treated as field separator. 

  live example: 

  770,"011",2002,2001247,3,4,"6/SOCJ,",1,0,20020131,122602,20020131,1,"FBLI"

  in 7th field, which is "6/SOCJ," the comma inside the quotes is mis-interpreted as field separator, and application submits error:  

  FGLEDX line 136090: expected 14 columns of data but found 15

  In case you need more information, please contact me at rafal@pies.pl. 

  Best regards, 

  Rafal Stanilewicz

  <hr><i>drh added on 2009-08-13 15:19:32:</i><br>
  The ".import" function is in the command-line shell, which is a separate
  program that is independent of the SQLite core.  I have retagged this
  ticket to indicate as much.

  There is no such thing as a "CSV" standard.  Different programs do different
  things.  There might be users that depend on the current behavior.  If we
  fix this for the OP, it could cause problems for others.  So it is unclear
  whether or not we should do anything for this.  We will simply leave the 
  ticket open for the time being.

  <hr><i>rogerb added on 2009-10-06 18:10:35:</i><br>
  See also [http://www.sqlite.org/cvstrac/tktview?tn=3812] where the shell isn't correctly importing the same data it exported.

  It would be reasonable to expect that whatever SQLite exports would import as exactly the same data.

  <hr><i>anonymous claiming to be PM added on 2009-10-14 11:15:47:</i><br>
  See also old ticket [http://www.sqlite.org/cvstrac/tktview?tn=3276].
  We still suffer from the inconsistency between the .import and export of the csv files. Although csv is not a real standard, at least you can expect consistent behaviour in sqlite (shell) itself.

  <hr><i>rogerb added on 2009-10-27 20:00:06:</i><br>
  A request to tweak some of the behaviour of .import such as continuing on errors

  [http://www.sqlite.org/cvstrac/tktview?tn=1506]

  <hr><i>rogerb added on 2009-10-27 21:17:53:</i><br>
  Header rows with .import [http://www.sqlite.org/cvstrac/tktview?tn=1313]

  <hr><i>anonymous added on 2010-01-06 15:37:44:</i><br>
  There is a well-defined standard for CSV files of mime type text/csv.  It can be found at:

  [http://tools.ietf.org/html/rfc4180]

  <hr><i>anonymous added on 2010-05-12 17:53:05:</i><br>
  Well, I need to import and export files from/to OpenOffice.org Calc (for example).

  I propose these modifications to make sqlite3 more easy to use (export/import with ".mode csv").

  In "src/shell.c" Version 3.6.23.1 :

  <pre>

  @@ -40,11 +40,11 @@
   
   #if defined(HAVE_READLINE) && HAVE_READLINE==1
   # include <readline/readline.h>
   # include <readline/history.h>
   #else
  -# define readline(p) local_getline(p,stdin)
  +# define readline(p) local_getline(p,stdin,0)
   # define add_history(X)
   # define read_history(X)
   # define write_history(X)
   # define stifle_history(X)
   #endif
  @@ -315,15 +315,16 @@
   ** fails.
   **
   ** The interface is like "readline" but no command-line editing
   ** is done.
   */
  -static char *local_getline(char *zPrompt, FILE *in){
  +static char *local_getline(char *zPrompt, FILE *in, const int bCsvMode){
     char *zLine;
     int nLine;
     int n;
     int eol;
  +  int bEscaped;
   
     if( zPrompt && *zPrompt ){
       printf("%s",zPrompt);
       fflush(stdout);
     }
  @@ -330,10 +331,11 @@
     nLine = 100;
     zLine = malloc( nLine );
     if( zLine==0 ) return 0;
     n = 0;
     eol = 0;
  +  bEscaped = 0;
     while( !eol ){
       if( n+100>nLine ){
         nLine = nLine*2 + 100;
         zLine = realloc(zLine, nLine);
         if( zLine==0 ) return 0;
  @@ -345,12 +347,15 @@
         }
         zLine[n] = 0;
         eol = 1;
         break;
       }
  -    while( zLine[n] ){ n++; }
  -    if( n>0 && zLine[n-1]=='\n' ){
  +    while( zLine[n] ){
  +      if( zLine[n]=='"' && bCsvMode ) bEscaped = 1 - bEscaped;
  +      n++;
  +    }
  +    if( !bEscaped && n>0 && zLine[n-1]=='\n' ){
         n--;
         if( n>0 && zLine[n-1]=='\r' ) n--;
         zLine[n] = 0;
         eol = 1;
       }
  @@ -367,11 +372,11 @@
   */
   static char *one_input_line(const char *zPrior, FILE *in){
     char *zPrompt;
     char *zResult;
     if( in!=0 ){
  -    return local_getline(0, in);
  +    return local_getline(0, in, 0);
     }
     if( zPrior && zPrior[0] ){
       zPrompt = continuePrompt;
     }else{
       zPrompt = mainPrompt;
  @@ -1625,25 +1630,45 @@
         sqlite3_finalize(pStmt);
         return 1;
       }
       sqlite3_exec(p->db, "BEGIN", 0, 0, 0);
       zCommit = "COMMIT";
  -    while( (zLine = local_getline(0, in))!=0 ){
  +    while( (zLine = local_getline(0, in, 1))!=0 ){
         char *z;
  +      char *q;
  +      int bEscaped;
  +      int bFieldQuoted;
  +      bEscaped = 0; /* to escape quotes */
         i = 0;
         lineno++;
  -      azCol[0] = zLine;
  -      for(i=0, z=zLine; *z && *z!='\n' && *z!='\r'; z++){
  -        if( *z==p->separator[0] && strncmp(z, p->separator, nSep)==0 ){
  +      if( *zLine=='"' ){
  +        azCol[0] = zLine + 1; /* ignore the first quote */
  +        bFieldQuoted = 1;
  +      }else{
  +        azCol[0] = zLine;
  +        bFieldQuoted = 0;
  +      }
  +      for(i=0, z=zLine, q=zLine; *z ; z++){
  +        if( *z=='"' ) bEscaped = 1 - bEscaped;
  +        if( bEscaped==0 && *z==p->separator[0] && strncmp(z, p->separator, nSep)==0 ){
  +          if( bFieldQuoted==1 && *q=='"') *q = 0; /* ignore the last quote */
             *z = 0;
             i++;
             if( i<nCol ){
               azCol[i] = &z[nSep];
  +            if( *azCol[i]=='"' ){
  +              azCol[i]++; /* ignore the first quote */
  +              bFieldQuoted = 1;
  +            }else{
  +              bFieldQuoted = 0;
  +            }
               z += nSep-1;
             }
           }
  +        q = z;
         } /* end for */
  +      if( bEscaped==0 && bFieldQuoted==1 && *q=='"' ) *q = 0; /* ignore the last quote */
         *z = 0;
         if( i+1!=nCol ){
           fprintf(stderr,
                   "Error: %s line %d: expected %d columns of data but found %d\n",
                   zFile, lineno, nCol, i+1);
  @@ -1651,10 +1676,16 @@
           free(zLine);
           rc = 1;
           break; /* from while */
         }
         for(i=0; i<nCol; i++){
  +        /* remove double quotes */
  +        for( bEscaped=0, z=azCol[i], q=azCol[i]; *z ; z++, q++){
  +          if( *z=='"' ) z++;
  +          *q = *z;
  +        }
  +        *q = 0;
           sqlite3_bind_text(pStmt, i+1, azCol[i], -1, SQLITE_STATIC);
         }
         sqlite3_step(pStmt);
         rc = sqlite3_reset(pStmt);
         free(zLine);

  </pre>


tkt_uuid: 93fb9f89d6a69dc3d8e627197d21a15dcb4eaaf7
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.6.17
resolution: Fixed
title: Index causes incorrect WHERE clause evaluation
comment:
  Consider the following SQL:

  <blockquote><pre>
  CREATE TABLE t1(x,y);
  INSERT INTO t1 VALUES(1,2);
  INSERT INTO t1 VALUES(2,7);
  INSERT INTO t1 VALUES(3,4);
  CREATE TABLE t2(a, b TEXT);
  CREATE INDEX t2b ON t2(b);
  INSERT INTO t2 VALUES(1,2);
  INSERT INTO t2 VALUES(5,6);
  SELECT *, y=b FROM t1, t2 WHERE y=b;
  </pre></blockquote>

  In the final SELECT statement, the last column ("y=b") shows zero (false) in 
  the output.  Clearly this must be wrong because if "y=b" is false, the WHERE
  clause should have prevented the row from being returned.

  The problem goes away if the index on t2(b) is removed.  It appears that
  the query planner is using the t2b index to resolve the WHERE clause
  even though the t1.y and t2.b columns
  have incompatible type affinities.

  <hr><i>drh added on 2009-08-19 15:21:05:</i><br>
  This error is present in all prior versions of SQLite back through
  version 3.1.3 (2005-02-19) and perhaps earlier.  (Version 3.1.3 is as far back as
  we looked.)

  The fix for the problem is check-in [e72186f2d68d28c2e0c32894f9adb28c155b5f63].

  Test cases for this fix are in the test/whereB.test test script.


tkt_uuid: 7d96113ff0aac1d54c96f9c0268fad1b6d757892
type: Build_Problem
status: Fixed
subsystem: Makefile
priority: Immediate
severity: Important
foundin: 3.6.17 + 7f4810747b
private_contact: 85acae05853464fa423d0d93c4179d8cc6572ab9
resolution: Fixed
title: SQLite fails to build when TCL isn't installed
comment:
  SQLite 3.6.17, with patch based on commit 7f4810747b0864981f27edbd504bfab2efea1e3c applied, fails to build when TCL isn't installed.

  <pre># make
  cat ./src/sqlite.h.in | tclsh ./tool/mksqlite3h.tcl `cat ./VERSION` > sqlite3.h
  /bin/sh: tclsh: command not found
  make: *** [sqlite3.h] Error 127</pre>

  <hr><i>drh added on 2009-08-16 18:03:57:</i><br>
  Here are some of my thoughts on requiring the use of tclsh in order to build
  SQLite from sources:

    1.  The recommended way of building SQLite is to use the amalgamation products
        that are made available with each release.  Tcl is not required to build
        the amalgamation.  Tcl is only required for building from the intermediate
        check-ins.

    2.  TCL has been installed by default on every Mac and on every version of
        Linux that I have ever used.  And installing TCL o windows is very
        simple.  Requiring TCL is not especially onerous.

    3.  SQLite began life as a TCL extension.  SQLite is a TCL extension that
        escaped into the wild.  It is fitting, therefore, for SQLite to pay homage
        to its progenitor.

  With these things in mind, I really don't have a problem with requiring
  TCL in order to build SQLite from canonical sources.


  <hr><i>anonymous claiming to be Arfrever@Gentoo.Org added on 2009-08-17 11:55:33:</i><br>
  The amalgamation tarball doesn't contain extensions, tests, TCL bindings etc.

  If the build-time dependency on TCL has to remain, then please update documentation (e.g. README) to state this.

  <hr><i>drh added on 2009-09-09 14:49:13:</i><br>
  README file updated by check-in [ba20091ae8146d276414f97527bf63e50fac8691].

  <hr><i>anonymous claiming to be Jeff Kowalczyk added on 2010-03-27 16:52:27:</i><br>
  I would respectfully ask that you reconsider the decision. For a source-based distribution such as Gentoo Linux, this imposes a unwanted dependency on tcl in order to use the desirable amalgamation build option.


tkt_uuid: fc6f41426befdba68558d6ca4da2dd2ba0b5116a
type: Incident
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.17
private_contact: 18f12f1b753b9f58ddf7127fd0af065d8602be5e
resolution: Not_A_Bug
title: update primary key failure
comment:
     create table test1(id integer primary key, msg text)
     update test1 set id=id+1 where id>100

  update returned "PRIMARY KEY must be unique"

  <hr><i>dan added on 2009-08-21 08:37:23:</i><br>
  This is because SQLite does not defer checking constraints until after the entire statement is executed. For example:

  <verbatim>
  SQLite version 3.6.17
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t(x INTEGER PRIMARY KEY);
  sqlite> INSERT INTO t VALUES(1);
  sqlite> INSERT INTO t VALUES(2);
  sqlite> UPDATE t SET x=x+1;  
  SQL error: PRIMARY KEY must be unique
  </verbatim>

  SQLite modifies the first row of the table, setting its primary key to 2. This clashes with the second row of the table and SQLite throws the "PRIMARY KEY must be unique" exception as a result. Even though if the statement were allowed to complete the final state of the database would not violate the constraint.

  This is, unfortunately for those who wish to execute such statements, how SQLite works. It is not considered a bug.


tkt_uuid: 2d401a94287b5cc5e74543626bc054377068ade1
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.17
resolution: Fixed
title: Segfault on unknown SQL function
comment:
  The following SQL results in an assertion failure or segfault:

  <verbatim>
     CREATE TABLE t1(x, y DEFAULT(nosuchfunc(1)));
     INSERT INTO t1(x) VALUES(2);
  </verbatim>

  <hr><i>drh added on 2009-08-18 16:12:28:</i><br>
  The problem has existing in the code ever since the ability to calls
  functions in  a DEFAULT clause was added in version 3.2.3, check-in
  [d273766ef2442eb87971b07f9ecf5682c88298ee] on 2005-06-03.

  Fixed by check-in [093917d7fda442012dfd1a1b2f20f85d2eefa093]


tkt_uuid: 92c5deed0700887e9c9726e56ebda34f22eb6c29
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.6.8
private_contact: 563b50dcd9670142d3c148c647f2021b18935f65
resolution: Fixed
title: FTS3 bug with multiple terms for exclusion
comment:
  Beginning in SQLite 3.6.8, a bug in FTS3 has been introduced, where the FTS3 search isn't processing multiple exclusion terms correctly.  SQLite 3.6.7 did behave as expected.

  For example, if I ran an FTS query for:
  black -blue -red

  FTS3 would not exclude all documents containing "blue" and "red".

  Below is a transcript of how the problem can be reproduced.


  $ cat test.sql
  create virtual table temp.test using fts3 (a text);
  insert into temp.test(a) values ('test testing');
  insert into temp.test(a) values ('test tester');
  insert into temp.test(a) values ('test fixed');
  insert into temp.test(a) values ('test testing tester');
  insert into temp.test(a) values ('test testing fixed');
  insert into temp.test(a) values ('test testing tester fixed');
  select * from test where a match 'test -fixed -tester';

  $ sqlite3-3.6.7 <test.sql
  test testing

  $ sqlite3-3.6.8 <test.sql
  test testing
  test fixed
  test testing fixed
  test testing tester fixed

  <hr><i>anonymous added on 2009-08-18 23:08:44:</i><br>
  Properly formatted transcript as follows:

  <pre>
  $ cat test.sql
  create virtual table temp.test using fts3 (a text);
  insert into temp.test(a) values ('test testing');
  insert into temp.test(a) values ('test tester');
  insert into temp.test(a) values ('test fixed');
  insert into temp.test(a) values ('test testing tester');
  insert into temp.test(a) values ('test testing fixed');
  insert into temp.test(a) values ('test testing tester fixed');
  select * from test where a match 'test -fixed -tester';

  $ sqlite3-3.6.7 <test.sql
  test testing

  $ sqlite3-3.6.8 <test.sql
  test testing
  test fixed
  test testing fixed
  test testing tester fixed
  </pre>

  <hr><i>dan added on 2009-08-19 14:52:28:</i><br>
  This bug was fixed for 3.6.17 here: [c19d419e8cf94a26d9bb6ad478e84841168a882e]

  <verbatim>
     dan@computer2:~/work/sqliteFE/bld$ cat test.sql
     create virtual table temp.test using fts3 (a text);
     insert into temp.test(a) values ('test testing');
     insert into temp.test(a) values ('test tester');
     insert into temp.test(a) values ('test fixed');
     insert into temp.test(a) values ('test testing tester');
     insert into temp.test(a) values ('test testing fixed');
     insert into temp.test(a) values ('test testing tester fixed');
     select * from test where a match 'test -fixed -tester';
     dan@computer2:~/work/sqliteFE/bld$ ./sqlite3
     SQLite version 3.6.17
     Enter ".help" for instructions
     Enter SQL statements terminated with a ";"
     sqlite> .read test.sql
     test testing
     sqlite>
  </verbatim>


tkt_uuid: 1248e6cda84b96e06819df267a3059ae46ddedcf
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.5.9
private_contact: 82840caba21e154009826fc7628a84df1daa541a
resolution: Rejected
title: Insufficient precision of sqlite3_column_text16() for floating point values
comment:
  sqlite3_column:text16() rounds floating point values with more than 15 decimal digits after the decimal point to 15 decimal digits after the decimal point.

  Here is the relevant code from "sqlite3VdbeMemStringify" which is called by "sqlite3_column_text16":

  }else{
      assert( fg & MEM_Real );
      sqlite3_snprintf(nByte, pMem->z, "%!.15g", pMem->r);
    }

  The precision of 15 is insufficient and should be 17 instead (see http://speleotrove.com/decimal/decifaq6.html#binapprox), otherwise what you store in the database is not exactly as what you get from the database (rounded to 15 decimal digits after the decimal point) and what you would get by sqlite3_column_double().


tkt_uuid: 2fd50fa6656950e19826089cd1abf19c35c26409
type: Feature_Request
status: Closed
subsystem: VFS
priority: Medium
severity: Important
foundin: <= 3.6.17
private_contact: cba955d6477c55010f038194a9e4fe6212be37aa
resolution: Fixed
title: cygwin-1.7 issues: winFullPathname and tempname
comment:
  I needed the attached patch for the new cygwin-1.7, 
  where cygwin_conv_to_full_win32_path() is deprecated.

  But I doubt that this patch is fully correct.
  winFullPathname is mainly used to get the 
  driveletter to check for the disksize
  and will fail for subst'ed drives.

  And I saw databases which were created at the cwd instead of the given path.

  Maybe someone sees the obvious mistake.


  2009-08-21 Reini Urban <rurbanb@x-ray.at>


          * os_win32.c (winFullPathname): support cygwin-1.7, fails with subst'ed drives

  <verbatim>
  diff -u sqlite-3.6.17/src/os_win.c.orig sqlite-3.6.17/src/os_win.c
  --- sqlite-3.6.17/src/os_win.c.orig	2009-08-08 19:44:58.000000000 +0200
  +++ sqlite-3.6.17/src/os_win.c	2009-08-21 09:07:33.906250000 +0200
  @@ -48,6 +48,7 @@
   
   #ifdef __CYGWIN__
   # include <sys/cygwin.h>
  +# include <cygwin/version.h>
   #endif
   
   /*
  @@ -1525,9 +1526,34 @@
     
   #if defined(__CYGWIN__)
     UNUSED_PARAMETER(nFull);
  +#if (CYGWIN_VERSION_API_MINOR >= 181)
  +  int nByte;
  +  char *zOut;
  +  WCHAR *zConverted;
  +  nByte = nFull + 260 + 1001; /* from the cygwin sources */
  +  zConverted = malloc( nByte*sizeof(zConverted[0]) );
  +  if( zConverted==0 ){
  +    return SQLITE_NOMEM;
  +  }
  +  cygwin_conv_path(CCP_POSIX_TO_WIN_W, zRelative, zConverted, nByte*sizeof(zConverted[0]));
  +#if 0
  +  printf("%s => %ls => %ls\n", zRelative, zConverted, &zConverted[4]);
  +#endif
  +  /* 4th: "\\?\D:..." => "D:..." */
  +  zOut = unicodeToUtf8(&zConverted[4]);
  +  free(zConverted);
  +  if( zOut ){
  +    sqlite3_snprintf(pVfs->mxPathname, zFull, "%s", zOut);
  +    free(zOut);
  +    return SQLITE_OK;
  +  }else{
  +    return SQLITE_NOMEM;
  +  }
  +#else
     cygwin_conv_to_full_win32_path(zRelative, zFull);
     return SQLITE_OK;
   #endif
  +#endif
   
   #if SQLITE_OS_WINCE
     UNUSED_PARAMETER(nFull);
  </verbatim>

  <hr><i>anonymous added on 2009-08-21 07:40:18:</i><br>
  sorry: wrong email
  correct is rurban@x-ray.at

  also sent to http://cygwin.com/ml/cygwin/2009-08/msg00698.html

  <hr><i>rogerb added on 2009-10-01 07:27:18:</i><br>
  See [http://www.sqlite.org/cvstrac/tktview?tn=3937] which includes some more coding related to this issue.


tkt_uuid: 43db771bb2fb8c9c17461d01ffed05aa874d20b4
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.17
private_contact: 082f5141404ab06cb5d77625e45f414db93d6a1b
resolution: Fixed
title: Sqlite shell prints some error messages to stdout
comment:
  The process_input function within shell.c writes most error messages to stderr, like the rest of the shell. But the error messages written out on line 2831 and line 2835 go to stdout. This makes it nearly impossible to reliably use the shell for non-interactive bulk / batch queries, and correctly detect and handle errors.

  Similarly, the timer stats are written to stdout on line 104 if enableTimer is on - it would be very useful to have these go to stderr so they're not commingled with data when operating in non-interactive mode but still available.

  <hr><i>shane added on 2009-10-21 03:55:12:</i><br>
  Check-in [8a8eeb0c5a]


tkt_uuid: efc02f9779194ed96405d5e42f569d71d576591c
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.17
private_contact: e3db43386dd373a0a7cb0e4e267f041be53229aa
resolution: Fixed
title: Trigger creation order affects query correctness
comment:
  I was able to boil down to a simple, reproducible test case.  For intuition: we create a table to hold all values (named "p") and then two tables ("four" and "five") to hold filtered subsets of of the data, automatically populated via trigger on insert.


  <verbatim>
  CREATE TABLE p (i INTEGER);
  CREATE TABLE four (val INTEGER);
  CREATE TABLE five (val INTEGER);
  CREATE TRIGGER five_trigger AFTER INSERT ON p WHEN NEW.i=5 BEGIN INSERT INTO five (val) VALUES (NEW.i); END;
  CREATE TRIGGER four_trigger AFTER INSERT ON p WHEN NEW.i=4 BEGIN INSERT INTO four (val) VALUES (NEW.i); END;
  INSERT INTO p (i) VALUES (5);

  SELECT * FROM five;
  </verbatim>

  On version SQLite 3.6.17 (downloaded today from sqlite.org, compiled on OS X 10.5.8) the final SELECT statement returns no rows (incorrect).  On version 3.4.0 (apparently pre-installed on OS X) the result is a single row with value of column "val" as the number 5 (correct). 

  If the order of the CREATE TRIGGER statements is reversed, I get the same (correct) behavior on both versions.  Sorry, but I haven't tried any version in between, nor variation on column data types or trigger operations.

  <hr><i>drh added on 2009-08-24 00:21:21:</i><br>
  Problem introduced by check-in [f099d6773a837dbe4ba85a8fda818e2d9466e743]
  on 2008-07-04 09:15:11.

  <hr><i>drh added on 2009-08-24 01:37:18:</i><br>
  Fixed by check-in [dee1b8eb402f47]


tkt_uuid: 2c5661c26a0c1b50e083af7110001267b934d912
type: Feature_Request
status: Closed
subsystem: Other
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: f4471882e74bca357ca3cf7b8709a92897657c2a
resolution: Rejected
title: Thread safety: does SQLite handle pthread_cancel cleanly?
comment:
  While fixing a threaded server application that uses sqlite3, I've notice that sometimes the application crashes during termination in sqlite functions depending on the timing of thread termination using pthread_cancel(). It is unclear to me whether sqlite takes pthread_cancel into consideration, so I thought to raise this ticket for future consideration, since I myself have been neglect until recently concerning issues with pthread_cancel.

  There are issues of concern related to pthread_cancel that I've come across recently:

  1. Calling pthread_cancel while a thread has a mutex locked, can cause other threads to deadlock. On OpenBSD (and probably Linux) pthread_cond_wait is a cancellation point, which means:

  pthread_lock(&mutex)
  while (condition not true)
    pthread_cond_wait(&cond_var, &mutex)
  pthread_unlock(&mutex)

  Could result in the thread terminating while blocked on a conditional variable  with mutex locked. I've found the following idiom useful:

  pthread_mutex_lock(&mutex)
  pthread_cleanup_push((void (*)(void *)) pthread_mutex_unlock, &mutex)
  while (condition not true)
    pthread_cond_wait(&cond_var, &mutex)
  pthread_cleanup_pop(1)

  in order to guarantee that the mutex is released when the thread terminates due to pthread_cancel. The use of pthread_cleanup_push / pop should probably be used any place that mutex lock/unlock protects a complex region of code, not just the case of conditional variables. OpenBSD documents their cancellation points, while Linux does not.

  From OpenBSD 4.0 man pthread_testcancel

     Cancellation Points
       Cancellation points will occur when a thread is executing the following
       functions: close(), creat(), fcntl(), fsync(), msync(), nanosleep(),
       open(), pause(), pthread_cond_timedwait(), pthread_cond_wait(),
       pthread_join(), pthread_testcancel(), read(), sigwaitinfo(),
       sigsuspend(), sigwait(), sleep(), system(), tcdrain(), wait(), waitpid(),
       write().

  2. Calling pthread_cancel while a thread is performing critical functions. If pthread_cleanup_push/pop are not suitable, then consider deferring thread cancellation around complex sections of code:

  int old_state;
  pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &old_state);
  ...long complex series of operations, like file IO...
  pthread_setcancelstate(old_state, NULL);

  I've found in my own code, in order to avoid potential problems related to pthread_cancel, that wrapping calls to sqlite3_exec, sqlite3_step, sqlite3_close appears to reduce, if not eliminate potential crashes and/or corruption.

  3. Another possible issue of concern pthread_setcanceltype, similar to pthread_setcancelstate, which can be set to deferred or asynchronous by the calling application. Library functions that temporarily disable cancellation probably need not worry. I raise the point only for consideration.


tkt_uuid: d2b46cfc75f6bfc6cae133bf442f0075b4fe4db5
type: Code_Defect
status: Closed
subsystem: Pager
priority: Immediate
severity: Critical
foundin: 3.6.17
private_contact: c47256bcd4810ab04846aa34d71b09638cdaf86b
resolution: Not_A_Bug
title: sqlite3BitvecClear will clear all bit in aHash when iSize>BITVEC_NBIT
comment:
  When I set, clear, test the bitvec object I found when the Bitvec.iSize>BITVEC_NBIT, the sqlite3BitvecClear will clear all bits in aHash table, is this normal?:

  <verbatim>
    Bitvec *pBitvec = 0;
    int sz=BITVEC_NBIT+1;
    int pos=10;
    int clearpos=20;

    pBitvec = sqlite3BitvecCreate( sz );
    sqlite3BitvecSet(pBitvec, pos);	 
    sqlite3BitvecClear(pBitvec, clearpos);
    rc=sqlite3BitvecTest(pBitvec,pos);
  </verbatim>

  rc should be 1, because the clear pos not equal to set pos, right? but it always return 0.

  <hr><i>drh added on 2009-08-24 12:17:14:</i><br>
  sqlite3BitvecClear() requires 3 parameters, not 2.  If you omit the buffer 
  in the third paramater, or if you supply a buffer that is too small, then
  you have violated the terms-of-use of sqlite3BitvecClear() and the behavior
  is undefined.  


tkt_uuid: 991789d9f3136a0460dc83a33e815c1aa9757c26
type: Code_Defect
status: Fixed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.6.17
private_contact: 6ee5271406387a8837880a2e933b17317c5a031a
resolution: Fixed
title:
  ext/fts? code uses tolower() when parsing tokens, which is problematic when using certain locales, such as tr_TR.UTF-8.

comment:
  We found this issue when we debugging a crash problem in chromium. See [http://code.google.com/p/chromium/issues/detail?id=15261] for detailed explanation.

  <hr><i>anonymous added on 2009-08-26 03:42:09:</i><br>
  You can find a patch here: [http://codereview.chromium.org/174387]


tkt_uuid: 4b81075908447e1e8117d72b3ced02a7744af144
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.16
private_contact: 814a4b3dad6ccc895b55631a7d8913381eee26ef
resolution: Not_A_Bug
title: In mult-thread that have a problem of updata lost
comment:
  When I use two thread too modified the database ,the later thread can't lock the database,when it lock the database and insert its data the prior thread insert data is lost.But in share cache model ,this problem do not occur  .
  That is my code:

  <verbatim>
  #include <pthread.h>
  #include <stdio.h>
  #include <sys/time.h>
  #include <string.h>
  #include "sqlite3.h" 
  #include   <time.h>    
  #include <stdlib.h> 
  #define MAX 2
  int k;
  pthread_t thread[MAX];
  sqlite3 *db[MAX];
  char *zErrMsg = 0;
  int rc;
  const char * sSQL3 = "select * from test where id=round(abs(random())/10000000000000);";
  const char * sSQL2 = "insert into test values(NULL,'siemens');";
  const char * sSQL1 = "BEGIN TRANSACTION;";
  const char * sSQL4 = "COMMIT;";
  const char * sSQL5 = "select count(*) from test;";
  static int callback(void *NotUsed, int argc, char **argv, char **azColName){
             int i;
             for(i=0; i<argc; i++){
               printf("%s = %s\n", azColName[i], argv[i] ? argv[i] : "NULL");
             }
             printf("\n");
             return 0;
            }

  void *thread1()
  {int temp,flag=0;
  temp=k;
      //     if ( sqlite3_enable_shared_cache(1)==SQLITE_OK ){
    //  printf( "shared _cache OK %d\n",temp);
      //sleep(2);
      //        };
              if(!temp)sleep(15);
              rc = sqlite3_open("./mydb.db", &db[temp]);
              if( rc ){
              fprintf(stderr, "Can't open database: %s\n", sqlite3_errmsg(db[temp]));
                sqlite3_close(db[temp]);
                return ;
                 }
          
          //  sleep(rand() % 10); 
              int i;
          sqlite3_exec(db[temp], sSQL1, callback, 0, &zErrMsg);
                  for (i=0;i<10;i++){
                   sleep(rand() % 5);
                  loop:
                   { 
                    rc=sqlite3_exec(db[temp], sSQL2, callback, 0, &zErrMsg);
                    if( rc!=SQLITE_OK ){
                       flag=1;
                        fprintf(stderr, "SQL error:%s\n", zErrMsg);
                        sleep(rand() % 10);
                            goto loop;
                             }
                          }
                         sqlite3_exec(db[temp], sSQL5, callback, 0, &zErrMsg);
                // printf("I am thread %d and %d times unlock the table\n",temp,i);
              
                  //  
                    }
                 
                    //}
              
              sqlite3_exec(db[temp], sSQL4, callback, 0, &zErrMsg);
              if(flag)printf("Thread %d has been encount DATABASE LOCKED",temp); 
               sqlite3_close(db[temp]);
          printf("thread %d:Is main thread are waiting me finish?\n",temp);
             pthread_exit(NULL);

  }

  void thread_create(void)
  {
          int temp, i;
          memset(&thread, 0, sizeof(thread));          
          /*Create thread */
          k=0;
          for(i=0;i<MAX;i++){
          if((temp = pthread_create(&thread[k], NULL, thread1, NULL)) != 0)   
                  printf("thread %d create failure\n",k);
          else
                printf("thread %d  create success\n",k);
                  sleep(1);
                  k++;
          }
          }
  void wait()
  {
          /*waiting for thread finish*/
          int i;
          for (i=0;i<MAX;i++){
          if(thread[i] !=0) {                   
                  pthread_join(thread[i],NULL);
               printf("thread %d finished \n",i);
          }
          
  }}
  int main()
  {


  int i; 
          time_t   t1,t2;    
           t1=time(NULL); 
          //printf("I am main thread ,I am creating thread \n");
          thread_create();
          printf("I am main thread, I am waiting for the thread \n");
           wait();
              // for(i=1;i<MAX;i++){
                //   sqlite3_close(db[i]);
                  // }
           t2=time(NULL);
           i=t2-t1;
           printf("Use time %d \n",i);
           return 0;
  }
  </verbatim>

  <hr><i>drh added on 2009-09-09 14:31:32:</i><br>
  Probably your COMMIT is failing - you do not seem to be checking the
  return code.  Seek help on the SQLite mailing list.  If this turns out
  to actually be a real problem, we will reopen the ticket.  But until
  evidence to the contrary emerges, we will assume this is a application
  error.


tkt_uuid: 03b198a2faeb91b0082ac364269f4185af65f9c4
type: Compiler_Warning
status: Fixed
subsystem: VFS
priority: Low
severity: Minor
foundin: 3.6.17
private_contact: 886478100d97cf8b9db98aa41afa84385aeaccf7
resolution: Fixed
title: Getting several warnings when compiling in eVC++ 4
comment:
  When compiling in Embedded Visual C++ 4.0 for HUW_MAXALL Win32 (WCE ARMV4) I get the following warnings. (for Microsoft Windows CE .Net v4.20)  

  <verbatim>
  c:\source path\sqlite3.c(11191) : warning C4244: '=' : conversion from '__int64 ' to 'unsigned long ', possible loss of data
  c:\source path\sqlite3.c(26767) : warning C4244: '=' : conversion from '__int64 ' to 'unsigned long ', possible loss of data
  c:\source path\sqlite3.c(26768) : warning C4244: '=' : conversion from '__int64 ' to 'unsigned long ', possible loss of data
  c:\source path\sqlite3.c(26924) : warning C4146: unary minus operator applied to unsigned type, result still unsigned
  c:\source path\sqlite3.c(26932) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(26941) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(26953) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(26963) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(26985) : warning C4146: unary minus operator applied to unsigned type, result still unsigned
  c:\source path\sqlite3.c(26992) : warning C4018: '>=' : signed/unsigned mismatch
  c:\source path\sqlite3.c(26993) : warning C4018: '<' : signed/unsigned mismatch
  c:\source path\sqlite3.c(27013) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(27021) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(27046) : warning C4018: '==' : signed/unsigned mismatch
  c:\source path\sqlite3.c(28077) : warning C4002: too many actual parameters for macro 'GetDiskFreeSpaceW'
  c:\source path\sqlite3.c(28052) : warning C4101: 'dwDummy' : unreferenced local variable
  </verbatim>

  I don't use any sqlite precomile definition besides:
  _WIN32_WCE=$(CEVersion),$(CePlatform),ARM,_ARM_,ARMV4,UNDER_CE=$(CEVersion),UNICODE,_UNICODE,NDEBUG

  The code compiles, with the same source, just fine in Visual Studio 2005 with the Windows Mobile kit.

  <hr><i>shane added on 2009-09-14 15:50:40:</i><br>
  Fixed several warnings when compiling at highest warning level under MSVC 2005.   See check-ins [f42ec993ac] and [1f0a93e17d]


tkt_uuid: 97214a34d814c288106c51d78dad08e457187e17
type: Code_Defect
status: Fixed
subsystem: VFS
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: c7290dda3501d52b13b2ab015f916b357301a2f6
resolution: Fixed
title: Use of os2MutexNotheld before their definition
comment:
  When compiling the amalgamation (within Mozilla) of 3.6.17 with SQLITE_DEBUG, one gets an error on OS/2 that os2Mutex[Not]Held is used before it's defined.

  <hr><i>anonymous added on 2009-08-28 07:38:33:</i><br>
  Hmm, I don't see a way to attach a patch any more, so I paste it here:

  <pre>
  ===========================================================================
  --- sqlite-3.6.17/src/mutex_os2.c	2009-06-12 04:37:47.000000000 +0200
  +++ sqlite-3.6.17/src/mutex_os2.c	2009-08-28 09:33:53.000000000 +0200
  @@ -176,7 +176,9 @@
     PID holder1;
     ULONG holder2;
     if( p==0 ) return;
  +#ifdef SQLITE_DEBUG
     assert( p->id==SQLITE_MUTEX_RECURSIVE || os2MutexNotheld(p) );
  +#endif
     DosRequestMutexSem(p->mutex, SEM_INDEFINITE_WAIT);
     DosQueryMutexSem(p->mutex, &holder1, &tid, &holder2);
     p->owner = tid;
  @@ -188,7 +190,9 @@
     PID holder1;
     ULONG holder2;
     if( p==0 ) return SQLITE_OK;
  +#ifdef SQLITE_DEBUG
     assert( p->id==SQLITE_MUTEX_RECURSIVE || os2MutexNotheld(p) );
  +#endif
     if( DosRequestMutexSem(p->mutex, SEM_IMMEDIATE_RETURN) == NO_ERROR) {
       DosQueryMutexSem(p->mutex, &holder1, &tid, &holder2);
       p->owner = tid;
  ===========================================================================
  </pre>

  Alternatives would be to move the definition of os2MutexHeld() and os2MutexNotheld() to the top of the file as in mutex_w32.c, or to use function prototypes at the top of the file.

  <hr><i>drh added on 2009-12-01 14:32:00:</i><br>
  See check-in [c40e4ef094bb9d58f14354602785ccc228f8bc2a].


tkt_uuid: 44a1b817c3feb2a9e49d437582e9702596443266
type: Incident
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.5
private_contact: 80d0d6a25903aacd9c1d3ed518757730af3717fb
resolution: Not_A_Bug
title: Problem with Kaspersky Anti-Virus and SQLite
comment:
  Our Application using a SQLite Database brings up exceptions on clients with the kaspersky anti virus program. Sometimes the exception Unable to oepn database is thrown by the application.
  Do you have any idea how to work around.

  <hr><i>drh added on 2009-09-01 15:23:50:</i><br>
  Please use tickets for reporting bugs against SQLite.  A bug is when
  SQLite malfunctions or computes the wrong answer.  This is a border-line
  case, admittedly.  This could be something wrong with SQLite.  But more
  likely, it is a problem with your anti-virus program or your application
  and you are asking for help in how to work around the problem.  For
  such requests, please post on the SQLite mailing list.

  If we determine that this really is a malfunction in SQLite, then we will
  reopen the ticket.  Until then, this ticket will be closed.


tkt_uuid: 9fd0bc36639c1513fe6bcef04f597bf859aa063c
type: Feature_Request
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: 22d2ebb9a6dbfa47bd4b558ba855d0331d912298
resolution: Fixed
title: Add a new open flag to enable shared cache for a single db connection
comment:
  We would like to be able to enable shared cache mode on a per-database connection basis.  Per our conversation today, adding a new flag that can be passed to sqlite3_open_v2() would be a great mechanism for doing so.

  <hr><i>drh added on 2009-09-09 14:26:04:</i><br>
  See check-in [f509eb8b488bb39deda4ca88f66ed40c6784af02].


tkt_uuid: 2ea2425d34bee9668ea8055016fed56aa5e773f8
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.6.17
private_contact: 5840dffc1663561cf96cb9eb1563447ad11d8bbe
resolution: Fixed
title: Assertion fault: Text encoding mismatch on value comparison
comment:
  The following SQL when run into an sqlite3 command-line shell (that is compiled
  with assert() enabled) will generate an assertion fault:

  <verbatim>
    PRAGMA encoding=UTF16;
    CREATE TABLE t1(a,b); 
    INSERT INTO t1 VALUES('abc',1);
    SELECT 1 FROM t1
     WHERE length(a) <= 10
       AND a <> 'xyz';
  </verbatim>

  The problem appears to have been introduced with version 3.5.8.
  The problem was discovered by Firefox developers as their bug number 514291.

  <hr><i>drh added on 2009-09-03 01:26:42:</i><br>
  Problem introduced by check-in [91cc646e2b0c1d] on 2008-04-01.
  Fixed by check-in [f22e388727f0] on 2009-09-03.


tkt_uuid: df2c3ad12a43eb7919ab5a98abf1806bdf2abb18
type: Code_Defect
status: Fixed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: SQLite version 3.6.14.2 with the CEROD Extension
private_contact: 6914eaa48fe1eae70e4a0f80eef2a3375a15fc04
resolution: Fixed
title: free() on unallocated memory when fts3 match phrase contains terms after "-" modifier
comment:
  Due to a (now corrected in my app UI) lack of filtering reserved chars from user search input, an internal error was exposed in FTS3, where memory that was not malloc'd is being free'd.

  Steps to reproduce:

  1. Create a table in a new, empty DB using FTS3.  A single column is sufficient:
     
  sqlite> create virtual table test using fts3 (text);

  2. Insert several records in the table created in (1):

  sqlite> insert into test values ("one two three");<br/>
  sqlite> insert into test values ("four five six");>br/>
  sqlite> insert into test values ("seven eight nine");<br/>

  3. Perform a select with a "match" where clause, having a term following an exclusion term (aka the '-' modifier):

  sqlite> select * from test where text match "four -two seven";
  sqlite3_cerod_shell(15877) malloc: *** error for object 0x10ed30: pointer being freed was not allocated
  *** set a breakpoint in malloc_error_break to debug

  The exception is occurring within the function sqlite3Fts3ExprFree() as it recursively descends Fts3Expr nodes.  Some of the nodes have left/right children that are not NULL but also not alloc'd.

  <hr><i>dan added on 2009-09-07 14:43:17:</i><br>
  This bug was fixed for 3.6.17 here: [http://www.sqlite.org/src/vinfo/c19d419e8cf94a26d9bb6ad478e84841168a882e|c19d419e8]

  So the easiest way to fix this problem is to upgrade to 3.6.17.

  <pre>
  dan@computer2:~/work/sqliteF/bld$ valgrind ./sqlite3 abc
  ==7004== Memcheck, a memory error detector.
  ==7004== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et al.
  ==7004== Using LibVEX rev 1804, a library for dynamic binary translation.
  ==7004== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
  ==7004== Using valgrind-3.3.0, a dynamic binary instrumentation framework.
  ==7004== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et al.
  ==7004== For more details, rerun with: -v
  ==7004== 
  SQLite version 3.6.18
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> create virtual table test using fts3 (text); 
  sqlite>  insert into test values ("one two three");
  sqlite> insert into test values ("four five six");   
  sqlite> insert into test values ("seven eight nine"); 
  sqlite> select * from test where text match "four -two seven";
  sqlite> ==7004== 
  ==7004== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 25 from 1)
  </pre>


tkt_uuid: 89f7b558545b9d92e7c19cdaf142e9b3ff53f1d3
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: any
private_contact: 8ea19cd970eababae356eb9b2fc21e0de2ad1baf
resolution: Rejected
title: RFE - Documentation in PDF format?
comment:
  Hi. Thanks for all the time and effort you put into SQLite.

  I see you are going to modify the documentation some time.

  Please could we have the user's documentation to SQLite in PDF document format?

  Things like:

  http://www.sqlite.org/quickstart.html

  http://www.sqlite.org/sqlite.html

  http://www.sqlite.org/lang.html

  http://www.sqlite.org/syntaxdiagrams.html


tkt_uuid: cbb5a315df99caacfbb1f066c9d153734bf9fbd3
type: Feature_Request
status: Closed
subsystem: Parser
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: eec320393e67955679760117f14dc8713a345cec
resolution: Rejected
title: Concatenating strings should be possible with a plus '+' operator
comment:
  The text concatenating operator ('||') is unsuitable for use with Visual Studio's Query Builder when using System.Data.SQLite.  Since many other SQL engines support using the plus ('+') operator to concatenate strings, SQLite should do the same.

  Currently I get the following with Visual Studio:
  "Error in SELECT clause: expression near '|'.
  Unable to parse query text."

  My query was:
  SELECT        EmployeeID, LastName, FirstName, FirstName || ' ' || LastName AS FullName
  FROM            Employees

  <hr><i>drh added on 2009-09-06 00:56:01:</i><br>
  Consider this statement:

     SELECT '1'+'2';

  As currently documented and implemented, SQLite gives an answer of 3.  You are
  asking that the answer be changed to '12'.

  SQLite has never worked this way.  And of the thousands and thousands of
  applications currently using SQLite, many would be broken if we changed the
  behavior now.  

  We must preserve backwards compatibility.  Therefore, this feature request
  is rejected.


tkt_uuid: e33452c90fc6fb9444776e40dc7d521ea52bc57d
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Low
severity: Critical
private_contact: 3299242f9eeb048063fa0161cd817ed02be44d65
resolution: Overcome_By_Events
title: Query plan doesn't use existing index
comment:
  <pre>
  sqlite> explain query plan select date(min(date_start)) from telephony_log;
  0|0|TABLE telephony_log
  sqlite> explain query plan select min(date_start) from telephony_log;
  0|0|TABLE telephony_log WITH INDEX telephony_log_date_start_idx ORDER BY
  </pre>


tkt_uuid: eaa105fe743a067384fd72409e76ea9bdfa70249
type: Feature_Request
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: 67bbaf11deabcf0a4a461858ed27b5c541d05d4e
resolution: Not_A_Bug
title: (...) WHERE [col] IN (val1) not reflected in xBestIndex
comment:
  I have created module implementing virtual table and registered it.

  When I query virtual table using WHERE col IN (?,? ...) 
  pIdxInfo->nConstraint passed to xBestIndex is equal zero, and therefore also argc in xFilter is 0 (col=? constraint is not reflected). I would expect that every value after IN will be reflected by separate 'equal' type record in pIdxInfo->aConstraint (i.e. above should be equivalent to WHERE col=? AND col=? ...).

  Sometimes it is not a problem as sqlite filters records out anyway, but in my case this leads to incorrect query results.
  This can be also observed by adding following test to vtab1.test:

  <verbatim>
  do_test vtab1-3.10-BUG {
    set echo_module ""
    execsql {
      SELECT * FROM t1 WHERE b IN (5,6);
    }
  } {4 5 6}

  do_test vtab1-3.11-BUG {
    set echo_module
  } [list xBestIndex {SELECT rowid, * FROM 'treal' WHERE b = ? AND b = ?}      \
          xFilter    {SELECT rowid, * FROM 'treal' WHERE b = ? AND b = ?} 5 6 ]
  </verbatim>

  vtab1-3.11-BUG will fail:

  Expected: |xBestIndex {SELECT rowid, * FROM 'treal' WHERE b = ? AND b = ?} xFilter {SELECT rowid, * FROM 'treal' WHERE b = ? AND b = ?} 5 6|

  Got: |xBestIndex {SELECT rowid, * FROM 'treal'} xFilter {SELECT rowid, * FROM 'treal'}|

  Regards,
  Grzegorz W.

  <hr><i>dan added on 2009-09-09 05:39:29:</i><br>
  Technically, "WHERE col IN (?,?)" is equivalent to "WHERE col=? OR col=?". If
  you rewrite the query as:

  <pre>
  SELECT * FROM t1 WHERE b=5 OR b=6
  </pre>

  then SQLite uses xBestIndex() to query the virtual table implementation to see
  if "WHERE b=?" can be optimized. If it can, SQLite may use this to optimize the
  query (depending on the estimated-cost returned by the vtab implementation for
  any other possible query plans.


  <hr><i>anonymous added on 2009-09-17 21:01:48:</i><br>
  Thats right, my mistake, of course it should be OR in an example.
  But problem still exist. In my application query is freely entered by user, so rewriting it to equivalent is not an option, and unfortunately I need information about constraints in xBestIndex/xFilter to return correct query result.


tkt_uuid: b2d64a52a0affa78fb66120700e1318bb98cb299
type: Build_Problem
status: Closed
subsystem: Makefile
priority: Immediate
severity: Minor
foundin: 3.6.16 (.17)
private_contact: 67bbaf11deabcf0a4a461858ed27b5c541d05d4e
resolution: Overcome_By_Events
title: pkgIndex.tcl is being deleted by 'make clean' but it is not created by 'make'
comment:
  Build scripts for TEA distribution:

  pkgIndex.tcl is being deleted by 'make clean' but it is not created by 'make'.
  This file is being created by ./configure, and should be deleted only by 'make distclean'.

  Fix is not difficult:

  - Add pkgIndex.tcl to CONFIG_CLEAN_FILES in Makefile.in

  - Remove it from CLEANFILES in configure.in

  Regards,
  Grzegorz W.


tkt_uuid: f76cabebfdbad11e92946336ec4daeae21c62354
type: Feature_Request
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: 67bbaf11deabcf0a4a461858ed27b5c541d05d4e
resolution: Rejected
title: xBestIndex/xFilter slight interface improvement
comment:
  I have created sqlite module which implements enum data type (i.e. value in db stored as int, but outside visible as string). I have started from test8.c echo module.

  To implement the feature I needed to pass list of column indexes specified in pIdxInfo->aConstraint (iColumn) from xBestIndex function to xFilter.
  And here is the problem. If I understand documentation correctly this is not guaranteed that xBestIndex and its connected xFilter calls will be one after one without any other calls with different parameters in between, so storing this data in some static data or so is not a good solution. The only legal way I see was to CONVERT my integer list to string and append to pIdxInfo->idxStr and then in xFilter function DECODE it, what is ugly and not efficient in my opinion.
  I could imagine that in more complicated modules one have to append in the same way whole configuration file and then will have to browse it in xFilter function.

  My proposition is to change definition of struct sqlite3_index_info (and therefore prototype of xFilter function) removing idxNum, idxStr, needToFreeIdxStr in charge of pointer to custom structure (void*) and pointer to function that is supposed to free memory of that pointer (prototype matching sqlite3_free() so that this function might be directly specified).
  If one pass NULL as free function pointer, this would mean that custom struct do not have to be freed by sqlite core.
  This kind of interface is already used in sqlite3_create_module_v2().

  As virtual tables interface is still marked as experimental, I hope this is not too late for such change, and all users would agree for adjusting their modules.

  Best Regards,
  Grzegorz W.

  PS: I may send source code of my module to clarify my point if necessary - just give me an e-mail request.

  <hr><i>drh added on 2009-09-09 13:18:51:</i><br>
  Your suggestion makes sense and would doubtlessly have been
  incorporated if it had appeared a few years ago.  But by this
  point, there are countless existing implementations of virtual
  tables.
  Changing the sqlite3_index_info structure would break much of that
  existing code, which is something we are loath to do.

  Note that sqlite3_index_info.idxStr does not necessarily have to be a 
  true string.
  It can be any data structure you want, as long as it can be freed using a
  single call to sqlite3_free().  There is no need to encode/decode your
  list of integers into a real string - just pass in the list of integers.

  <hr><i>anonymous added on 2009-09-17 21:22:06:</i><br>
  That's shame. Actually I needed both SQL string and index array, and what really is missing is custom function to free memory in case of more complex structures.

  But there is probably yet better workaround: just place all 'structures' produced by xBestIndex on some dynamically growing array or list remembered by module, and pass index (or pointer) to recently added item in pIdxInfo->idxNum or pIdxInfo->idxStr (setting needToFreeIdxStr=0). Then recursively free entire list/array in xFilter (or in xDisconnect). This probably should work correctly at least if connection is used in one thread.


tkt_uuid: 3350da7d35de093b00d14ca8bd2200dd86d97415
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.3
private_contact: cb12c91fed1958432cd984aebd5061ab778f53a6
resolution: Not_A_Bug
title: Error: malformed database schema - near "VIRTUAL": syntax error
comment:
  <verbatim>
  by connecting sqlite2 1_Mix.db 
  sqlite3>.tables
  Error: malformed database schema - near "VIRTUAL": syntax error
  </verbatim>

  <hr><i>drh added on 2009-09-09 12:50:52:</i><br>
  There are many possible reasons why the OP might be getting the error message
  he is getting.  Without more information, it is impossible to isolate the
  problem.  We can say, however, that the cause is like to be database corruption
  or a miscompiled copy of SQLite rather than a bug in SQLite.

  A "bug" is when SQLite crashes or gets the wrong answer.  The description above
  does not establish that this is a bug.  More likely, this is a corrupt database.

  The OP should ask for help on the SQLite mailing list.  If it turns out this
  really is a bug, then we will reopen the ticket.


tkt_uuid: 29ab7be99f2cfe0b04466dd95b80ace87e9fc1b2
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.17
resolution: Fixed
title: Hitting NOT NULL constraint does not roll back statement transaction
comment:
  Running the following in 3.6.17:

  <verbatim>
  do_test haltifnull-1.1 {
    execsql {
      CREATE TABLE t1(a, b NOT NULL);
      CREATE TABLE t2(c, d);
      INSERT INTO t2 VALUES(3, 4);
      INSERT INTO t2 VALUES(5, NULL);
    }
  } {}

  do_test haltifnull-1.2 {
    catchsql {
      BEGIN;
      INSERT INTO t1 VALUES(1, 2);
      INSERT INTO t1 SELECT * FROM t2;
    }
  } {1 {t1.b may not be NULL}}

  do_test haltifnull-1.3 {
    execsql { SELECT * FROM t1 }
  } {1 2}
  </verbatim>

  The SELECT in test case 1.3 returns two rows - "1 2" and "3 4". It should only return 1 - "1 2".

  The problem is that the "INSERT INTO t1 SELECT * FROM t2" is not opening a
  statement transaction. So change made prior to hitting the NOT NULL constraint
  (inserting the "3 4" row) is not being rolled back as it should be.

  <hr><i>dan added on 2009-09-09 11:47:56:</i><br>
  Introduced here [/info/feccad8d0d|feccad8d0d] (sqlite version 3.6.11). Fixed as part of trigger-related changes for 3.6.18.


tkt_uuid: 92bd6eaf04e11711f14cb8b3bebb615ae0267550
type: Build_Problem
status: Closed
subsystem: Makefile
priority: Immediate
severity: Cosmetic
foundin: 3.6.17
private_contact: 7d648926689058d41bb399cda295099d354b5b7d
resolution: Fixed
title: Suggestion to revise clean rule in main.mk.
comment:
  On Cygwin and MinGW, second rebuld by using main.mk will be faild.

  <verbatim>
  $ make -f Makefile.linux-gcc libsqlite3.a
      (The first building will suceed.)
  $ make -f Makefile.linux-gcc clean
  $ make -f Makefile.linux-gcc libsqlite3.a
      (The second rebuilding will failed.)
  gcc -g -O2 -o mkkeywordhash -DNDEBUG=1 -
  DHAVE_FDATASYNC=1 ../sqlite/tool/mkkeywordhash.c
  ./mkkeywordhash >keywordhash.h
  cp ../sqlite/src/parse.y .
  rm -f parse.h
  ./lemon -DNDEBUG=1 -DHAVE_FDATASYNC=1 parse.y
  Can't open the template file "lempar.c".
  make: *** [parse.c] Error 1
  $ rm lemon.exe mkkeywordhash.exe 
  $ make -f Makefile.linux-gcc libsqlite3.a
      (Rebuilding will suceed.)
  </verbatim>

  When you read, please replace ``foo'' to ``lemon'' or ``mkkeywordhash''.
  On Cygwin and MinGW, foo.exe will be generated by following rule.

  <verbatim>
  foo: foo.c
  	$(CC) -o $@ $<
  </verbatim>

  But, following ``clean'' rule can not delete foo.exe.

  <verbatim>
  clean:
  	$(RM) foo
  </verbatim>

  So, I suggest to revise some rules for executable files as following.

  <verbatim>
  foo$(EXE): foo.c
  	$(CC) -o $@ $<

  clean:
  	$(RM) foo$(EXE)
  </verbatim>


tkt_uuid: 6e1086fb2ccc8191017a42768e04262a535b2125
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: 0b3aff16612e1e9f3b675963fa64e01bd1d89307
resolution: Not_A_Bug
title: "create table as select" and "pragma *_column_names"
comment:
  Table copies get wrong column names when certain pragma settings are
  made. The following SQLite shell sequences shows this:

  <verbatim>
  > create table foo(id int, name text);
  > pragma full_column_names=on;
  > pragma short_column_names=off;
  > create table foo_copy as select * from foo;
  > pragma table_info(foo)
  0|id|int|0||0
  1|name|text|0||0
  > pragma table_info(foo_copy);
  0|foo.id|INT|0||0
  1|foo.name|TEXT|0||0
  </verbatim>


tkt_uuid: 65bbb65a47ce115ecb0ea9e1c64c65fb8628f327
type: Build_Problem
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.18
private_contact: 85acae05853464fa423d0d93c4179d8cc6572ab9
resolution: Fixed
title: 'configure' file without +x permission in sqlite-3.6.18.tar.gz tarball
comment:
  'configure' file doesn't have +x permission in sqlite-3.6.18.tar.gz tarball. I can fix permissions manually, but I would like to suggest that you ensure that tarballs of newer versions contain files with correct permissions. I suspect that this bug might have been introduced during recent transition to Fossil.

  <hr><i>drh added on 2009-09-11 18:04:37:</i><br>
  See check-in [5e0e941f3adf09c5].

  Fossil is suppose to track execute permission, but something isn't working
  quite right with that at the moment.  We'll fix fossil separately.


tkt_uuid: 9a502b7a35008547632cbc6fcd07d8f55afbcdfe
type: Documentation
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.10
private_contact: 27135620fb4170e20e132b0e99a646f4029c5c0c
resolution: Fixed
title: datetime function has year 54079 bug
comment:
  datetime() function with a julian day argument gives strange results for dates before JD -0.5 (-4713-11-24 00:00:00) or after JD 21473068.49999999 (54079-02-28 23:59:59).

  sqlite> select datetime(-0.5001);
  -4713-11-24 00:00:-8

  sqlite> select datetime(21473068.5);
  54079-1403570-15 00:00:00

  <hr><i>drh added on 2009-09-12 04:06:08:</i><br>
  Documentation has been updated to state that dates outside the range
  of 0000-01-01 through 9999-12-31 are not supported.
  See [http://www.sqlite.org/docsrc/vinfo/62853a019650].


tkt_uuid: ab7498f2f66ecb6ead664456e5a72c66a8506614
type: Performance_Issue
status: Closed
subsystem: Unknown
priority: Low
severity: Important
foundin: 3.6.16
private_contact: 718c83cc9eb92cdd77f8fce1ae6fa7b90b433993
resolution: Overcome_By_Events
title:
  54 seconds to sort two(2) rows only when LIKE operator is used in WHERE clause and the query also contains an order-by clause

comment:
  Notice that in the six queries below, performance is blisteringly fast UNLESS both of the following conditions exist, in which case it is dead-dog slow:
  a) a LIKE operator is used in the WHERE clause
  b) the query contains an ORDER-BY clause that references columns defined in the composite primary key CONSTRAINT on the WRDS table, and those columns have primacy in the order-by column sequence

  1. EXECUTION TIME FOR THIS QUERY: 1.6 seconds
  select  count(*)
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  like 'hlanc%'  and not(wrd.forn = 1)

  2. EXECUTION TIME FOR THIS QUERY: 0.6 seconds
  select  wrds.ttl,wrds.pos,wrds.wrdid,wrds.edln,wrds.corn,wrd.forn,wrds.rune,wrds.crlf,wrd.spelling  
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  like 'hlanc%'  and not(wrd.forn = 1)

  3. EXECUTION TIME FOR THIS QUERY: 0.04 seconds
  select  wrds.ttl,wrds.pos,wrds.wrdid,wrds.edln,wrds.corn,wrd.forn,wrds.rune,wrds.crlf,wrd.spelling  
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  = 'hlanca'  and not(wrd.forn = 1)
  order by wrds.ttl, wrds.pos, wrds.wrdid

  3. EXECUTION TIME FOR THIS QUERY: 1.9 seconds
  select  wrds.ttl,wrds.pos,wrds.wrdid,wrds.edln,wrds.corn,wrd.forn,wrds.rune,wrds.crlf,wrd.spelling  
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  LIKE 'hlanc%'  and not(wrd.forn = 1)
  order by spelling

  5. EXECUTION TIME FOR THIS QUERY: 0.6 seconds
  select  wrds.ttl,wrds.pos,wrds.wrdid,wrds.edln,wrds.corn,wrd.forn,wrds.rune,wrds.crlf,wrd.spelling  
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  LIKE 'hlanc%'  and not(wrd.forn = 1)
  order by spelling, wrds.ttl, wrds.pos


  6. EXECUTION TIME FOR THIS QUERY: 54.9 seconds !!
  select  wrds.ttl,wrds.pos,wrds.wrdid,wrds.edln,wrds.corn,wrd.forn,wrds.rune,wrds.crlf,wrd.spelling  
  FROM WRD INNER JOIN WRDS on WRD.WRDID=WRDS.WRDID INNER JOIN TTL on WRDS.TTL=TTL.TTL 
  WHERE ttl.cat In ( 'P' , 'V' , 'G' )  AND wrd.spelling  LIKE 'hlanc%'  and not(wrd.forn = 1)
  order by wrds.ttl, wrds.pos, wrds.wrdid


  Table WRDS is defined as follows (and contains > 3.5 million rows):


  CREATE TABLE [WRDS] (
      [ttl] int NOT NULL,
      [pos] int NOT NULL,
      [wrdid] int NOT NULL,
      [edln] varchar(8),
      [corn] bit,
      [rune] bit,
      [crlf] bit,
      CONSTRAINT [PK_WRDS] PRIMARY KEY ([ttl], [pos], [wrdid])
  );
  CREATE INDEX [IX_WRDID] ON [WRDS] ([wrdid]);

  I believe the optimizer may be using the tacit index that implement the composite primary key constraint on this table, rather than the raw result rows returned by the query, even if it would be much less costly to ignore the index. ***But this bad decision appears to be made only only when the LIKE operator is used in the WHERE clause.***

  The TTL table contains appox 3,000 rows.
  The WRD table contains approx 263,000 rows.
  the WRDS table contains approx 3,770,000 rows.

  Thanks for looking into this situation. It would force me to keep my app in Microsoft Access rather than porting it to SQLite.
  Tim Romano


tkt_uuid: 8c743979c2450f91eb43901ec61017bac7f01bd3
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.16
private_contact: 718c83cc9eb92cdd77f8fce1ae6fa7b90b433993
resolution: Not_A_Bug
title: SQLite version correction on LIKE operator + ORDER BY bug I reported earlier today
comment:
  I am using version 3.6.16 that ships with the ADO.NET provider written by Robert Simpson.  I had written 3.1.16 in error.

  http://sqlite.phxsoftware.com/


tkt_uuid: 127c1c42ff7c95312a8194f27249703c7ab1b238
type: Build_Problem
status: Closed
subsystem: Makefile
priority: Immediate
severity: Critical
foundin: 3.6.18
private_contact: 3299242f9eeb048063fa0161cd817ed02be44d65
resolution: External_Bug
title: Build error "C compiler cannot create executables" on debian lenny host
comment:
  <pre>
  chmod a+x /srv/sqlite3/sqlite3-3.6.18/./configure
  cd . && CC="cc" CXX="g++" CFLAGS="-g -O2 -g -Wall -O2 
  -fno-strict-aliasing -DSQLITE_CORE=1 -DSQLITE_ENABLE_COLUMN_METADATA -
  DSQLITE_ENABLE_STAT2 _DSQLITE_DEFAULT_RECURSIVE_TRIGGERS=1 -
  DSQLITE_ENABLE_COMPRESS -DSQLITE_ENABLE_ENV -DSQLITE_ENABLE_FTS3 -
  DSQLITE_ENABLE_INET -DSQLITE_ENABLE_MD5 -DSQLITE_ENABLE_RTREE -
  DSQLITE_ENABLE_TABLEFUNC -DSQLITE_ENABLE_UNICODE -DSQLITE_ENABLE_UUID -
  DSQLITE_ENABLE_VIRTUALTEXT -DSQLITE_ENABLE_KEY -DSQLITE_ENABLE_TEMPKEY -
  DSQLITE_ENABLE_UNDO -DSQLITE_ENABLE_VERSIONING -DSQLITE_SOUNDEX -
  DSQLITE_TEMP_STORE=3 -DSQLITE_DEFAULT_PAGE_SIZE=4096 -
  DSQLITE_DEFAULT_CACHE_SIZE=32000 -DSQLITE_DEFAULT_TEMP_CACHE_SIZE=8000 -
  DSQLITE_DEFAULT_FILE_FORMAT=4" CXXFLAGS="-g -O2 -g -Wall -O2 -fno-strict-
  aliasing -DSQLITE_CORE=1 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_STAT2 
  _DSQLITE_DEFAULT_RECURSIVE_TRIGGERS=1 -DSQLITE_ENABLE_COMPRESS -
  DSQLITE_ENABLE_ENV -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_INET -
  DSQLITE_ENABLE_MD5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_TABLEFUNC -
  DSQLITE_ENABLE_UNICODE -DSQLITE_ENABLE_UUID -DSQLITE_ENABLE_VIRTUALTEXT -
  DSQLITE_ENABLE_KEY -DSQLITE_ENABLE_TEMPKEY -DSQLITE_ENABLE_UNDO -
  DSQLITE_ENABLE_VERSIONING -DSQLITE_SOUNDEX -DSQLITE_TEMP_STORE=3 -
  DSQLITE_DEFAULT_PAGE_SIZE=4096 -DSQLITE_DEFAULT_CACHE_SIZE=32000 -
  DSQLITE_DEFAULT_TEMP_CACHE_SIZE=8000 -DSQLITE_DEFAULT_FILE_FORMAT=4" 
  CPPFLAGS="" LDFLAGS="-luuid -lz -ldl" /srv/sqlite3/sqlite3-3.6.18/./configure 
  --build=i486-linux-gnu --prefix=/usr --includedir="\${prefix}/include" --
  mandir="\${prefix}/share/man" --infodir="\${prefix}/share/info" --
  sysconfdir=/etc --localstatedir=/var --libexecdir="\${prefix}/lib/sqlite3" --
  disable-maintainer-mode --disable-dependency-tracking --srcdir=.  --with-
  tcl=/usr/lib/tcl8.5 --enable-threadsafe --enable-load-extension
  checking build system type... i486-pc-linux-gnu
  checking host system type... i486-pc-linux-gnu
  checking for gcc... cc
  checking for C compiler default output file name...
  configure: error: C compiler cannot create executables
  See `config.log' for more details.
  make: *** [config.status] Ошибка 77
  </pre>

  <hr><i>anonymous added on 2009-09-13 12:27:00:</i><br>
  This has nothing to do with SQLite - it is an issue with your system.  Did you look in config.log as the error output says?

  <hr><i>anonymous added on 2009-09-13 20:15:35:</i><br>
  The config.log is not useful:

  <pre>
  tail -f config.log
  ## confdefs.h. ##
  ## ----------- ##

  #define PACKAGE_NAME "sqlite"
  #define PACKAGE_TARNAME "sqlite"
  #define PACKAGE_VERSION "3.6.18"
  #define PACKAGE_STRING "sqlite 3.6.18"
  #define PACKAGE_BUGREPORT ""

  configure: exit 77
  </pre>

  <hr><i>anonymous added on 2009-09-13 20:16:55:</i><br>
  I don't think that this is external bug because previous version compiled success. Makefile of new release is incorrect.

  <hr><i>anonymous added on 2009-09-14 04:34:59:</i><br>
  This is still not an issue with SQLite no matter how desperately you want it to be.  configure error 77 is fairly common:

    http://www.google.com/search?q=configure+exit+77

  <hr><i>anonymous added on 2009-09-14 06:38:42:</i><br>
  It isn't even repeatable.  I have a freshly installed Lenny but with the compiler installed in order to make Python extensions.  configure works just fine.

  As your output originally indicated *you* have a problem on *your* where the compiler doesn't produce executables, is incompletely installed or not installed at all.  *You* need to fix that.  There is no magic SQLite can do to fix it.

  <hr><i>anonymous added on 2009-09-14 06:52:52:</i><br>
  I have debian lenny chroot for build SQLite. All previous versions are buildable and so all dependencies are installed. But new release is not buildable. Why?

  <hr><i>anonymous added on 2009-09-15 10:13:44:</i><br>
  We can't see your chroot from here and it doesn't match a real fresh install.  Given your configure output shows the compiler being detected as cc and not gcc, it is almost 100% certain you have done something so that the compiler isn't installed, is wrongly installed or you experimented with a different one.


tkt_uuid: 4abd1e4c003842774a00f4f1431b8d798a60ef5a
type: Documentation
status: Fixed
subsystem: Documentation
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 9d6856406b42adedf84fe1e8b8496edb64a9f1d6
resolution: Fixed
title: omitted.html still says recursive triggers are not supported
comment:
  The title says it all.
  Please update that webpage so Sqlite gets full credit for what it can do.

  <hr><i>drh added on 2009-12-08 14:43:11:</i><br>
  This has been fixed.  Not sure when the fix was checked in, but the website
  is not correct.


tkt_uuid: ff44d82f3be086193da3a833a0ff28ca03336986
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: fts3ReallocOrFree and fts3InitVtab are not marked static
comment:
  In the amalgamation fts3ReallocOrFree is not marked as static/private and should be.

  <hr><i>rogerb added on 2009-12-09 04:15:26:</i><br>
  In 3.6.21 fts3InitVtab is now also not marked static or SQLITE_API (regression)


tkt_uuid: 806f3be2d7773468bb587031b58e185c11ae1022
type: Feature_Request
status: Closed
subsystem: B-Tree
priority: Immediate
severity: Cosmetic
foundin: 3.6.18
private_contact: ea1ef70cfca177467682af5cf7c21b8b666106b4
resolution: Rejected
title: Proposal to inline some functions
comment:
  There's no problem.
  I would like to propose inlining of several pretty often used functions:
   sqlite3BtreeGetPageSize
   sqlite3BtreePager
   sqlite3PagerBackupPtr
   sqlite3PagerRef
   sqlite3PagerPageRefcount
   sqlite3PagerGetData
   sqlite3PagerGetExtra
   sqlite3PagerIsreadonly
   sqlite3PagerRefcount
   sqlite3PagerFilename
   sqlite3PagerVfs
   sqlite3PagerFile
   sqlite3PagerJournalname
   sqlite3PagerNosync
   sqlite3PagerTempSpace
   sqlite3PagerIsMemdb
  For instance, I introduced a compiler flag named INLDEF and for each ot these functions I issued:
  in the declaration section:

  <verbatim>
      #ifdef INLDEF
      # define sqlite3BtreeGetPageSize(a) (a)->pBt->pageSize
      #else//INLDEF
      SQLITE_PRIVATE INL int sqlite3BtreeGetPageSize(Btree*);
      #endif//INLDEF
  </verbatim>

  and in the implementation section: 

  <verbatim>
      #ifndef INLDEF
      SQLITE_PRIVATE int sqlite3BtreeGetPageSize(Btree *p){
        return p->pBt->pageSize;
      }
      #endif//INLDEF
  </verbatim>

  Best regards: Chris

  <hr><i>drh added on 2009-09-14 15:35:28:</i><br>
  Modern C compilers do this kind of inlining automatically.  There is no need
  to complicate the code.


tkt_uuid: a696379c1f0886615541a48b35bd8181a80e88f8
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 24a18e1e5d4563bf74c495363cc2475f7f44905e
resolution: Fixed
title: Access violation error trying to insert into triggered view
comment:
  i've the following DB (these are the scripts):
  <verbatim>
  CREATE TABLE [RECETAS] (
  [RECETA_ID] INTEGER  PRIMARY KEY AUTOINCREMENT NULL,
  [NOMBRE] VARCHAR(128)  NULL,
  [DESCRIPCION] VARCHAR(1024)  NULL,
  [TEXTO] TEXT  NULL,
  [IMAGE_ID] INTEGER  NULL,
  [PUNTAJE] INTEGER  NULL,
  [DATE_CREATE] TIMESTAMP  NULL,
  [DATE_EDIT] TIMESTAMP  NULL
  );

  CREATE TABLE [prueba] (
  [campo1] VARCHAR(10124)  NULL
  );

  CREATE VIEW [vw_datos] AS 
  select * from prueba;

  CREATE TRIGGER [ON_TBL_PRUEBA_] 
  AFTER DELETE ON [prueba] 
  FOR EACH ROW 
  BEGIN 

  select 1;

  END;

  CREATE TRIGGER [ON_1] 
  BEFORE UPDATE ON [prueba] 
  FOR EACH ROW 
  BEGIN 

  select 1;

  END;

  CREATE TRIGGER [ON_VW_DATOS_2] 
  INSTEAD OF INSERT ON [vw_datos] 
  FOR EACH ROW WHEN NEW.CAMPO1='PROBANDO' 
  BEGIN 

  select 1;

  END;

  CREATE TRIGGER [ON_VW_DATOS_1] 
  INSTEAD OF INSERT ON [vw_datos] 
  FOR EACH ROW 
  BEGIN 

  insert into recetas (nombre) values(new.campo1);

  END;
  </verbatim>

  when i try to insert a row into the view i get an error like this one:

  Access violation at address 6091167E in module 'sqlite3.dll'. Read of address 52011B07  <insert into vw_datos values("dddd")>

  <hr><i>drh added on 2009-09-14 22:28:42:</i><br>
  The following script demonstrates the bug more compactly:

  <verbatim>
     CREATE TABLE t1(x INTEGER PRIMARY KEY AUTOINCREMENT, y);
     CREATE TABLE t2(z);
     CREATE VIEW v1 AS SELECT * FROM t2;
     CREATE TRIGGER r1 INSTEAD OF INSERT ON v1 BEGIN
       INSERT INTO t1(y) VALUES(new.z);
     END;
     INSERT INTO v1 VALUES(0);
  </verbatim>

  The problem first introduced sometime after [dee1b8eb40] (2009-08-24), and
  on or before check-in [8a746fbfd5] (2009-08-31).  The precise origin of the
  problem cannot be more precisely localized.  This band of changes are where
  the trigger logic was refactored to support recurse triggers.
   

  <hr><i>drh added on 2009-09-15 00:03:19:</i><br>
  Fixed by check-in [2b2a1ef74e24857b8d18f6370894588fc2aa5ebb]


tkt_uuid: 084941461fc67dc9eb41d625847a30abed5c2eb0
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Overcome_By_Events
title: Some minor asyncvfs issues
comment:
  The symbol asyncFileSize should be declared static. 

  In the header file lines 78 and 97 contain prototypes but there are no paramaters so technically they aren't prototypes.  void should be placed between the parentheses.

  It would be nice if the 4 public functions making up the API were prefixed with a symbol like SQLITE3ASYNCVFS_API which is defined to nothing by default.  Windows folks can then make it declspec(export) and I'd like to make it static.

  <hr><i>anonymous claiming to be rogerb@rogerbinns.com added on 2009-09-16 10:26:06:</i><br>
  The vfs structure is also messed up.  It is missing the xGetLastError member - also see http://www.sqlite.org/cvstrac/tktview?tn=3337

  The last 3 lines have a mismatch between the comments and the functions:

  <pre>
      asyncRandomness,      /* xDlError */
      asyncSleep,           /* xDlSym */
      asyncCurrentTime      /* xDlClose */
  </pre>


tkt_uuid: 832b2f3e6e0c61a6aa4c5dfea8915fbb36d40a11
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.18
private_contact: 3299242f9eeb048063fa0161cd817ed02be44d65
resolution: External_Bug
title: integrity_check error after backup restore
comment:
  $ sqlite3 work.db.prev '.dump'>work.db.sql

  $ sqlite3 work.db '.read work.db.sql'

  $ sqlite3 work.db

  SQLite version 3.6.18

  Enter ".help" for instructions

  Enter SQL statements terminated with a ";"

  sqlite> pragma integrity_check;

  rowid 1 missing from index sqlite_autoindex_groups_1

  <hr><i>anonymous added on 2009-09-16 06:55:57:</i><br>
  <pre>
  $ sqlite3 /tmp/test.db
  SQLite version 3.6.18
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> BEGIN TRANSACTION;
  sqlite> CREATE TABLE groups
     ...> (
     ...>         name TEXT collate NOCASE UNIQUE NOT NULL DEFAULT ''
     ...> );
  SQL error: table groups already exists
  sqlite> INSERT INTO "groups" VALUES('Администраторы');
  SQL error: column name is not unique
  sqlite> CREATE TABLE const_telephony_direction (
     ...>         name text not null,
     ...>         destcode text not null
     ...> );
  SQL error: table const_telephony_direction already exists
  sqlite> INSERT INTO "const_telephony_direction" VALUES('Совинтел_МГМН_старые_кмст','7485');
  sqlite> CREATE INDEX const_telephony_direction_complex_idx on const_telephony_direction(name,destcode);
  SQL error: index const_telephony_direction_complex_idx already exists
  sqlite> COMMIT;
  sqlite>
  sqlite> pragma integrity_check;
  rowid 1 missing from index sqlite_autoindex_groups_1
  </pre>

  <hr><i>anonymous added on 2009-09-16 07:16:08:</i><br>
  The problem is redefined NOCASE collation. This unicode support collation can be obtained from 

  http://mobigroup.ru/files/sqlite-ext/unicode/

  How to fix it for new SQLite versions?

  <hr><i>anonymous added on 2009-09-16 07:40:44:</i><br>
  The problem can be temporary fixed by removing indexes by columns with redefined NOCASE collation (groups.name autoindex and composite index with const_telephony_direction.name):

  <pre>$ sqlite3 :memory:
  SQLite version 3.6.18
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> BEGIN TRANSACTION;
  sqlite> CREATE TABLE groups
     ...> (
     ...>         name TEXT collate NOCASE UNIQUE NOT NULL DEFAULT ''
     ...> );
  sqlite> INSERT INTO "groups" VALUES('Администраторы');
  sqlite> CREATE TABLE const_telephony_direction (
     ...>         name text not null,
     ...>         destcode text not null
     ...> );
  sqlite> INSERT INTO "const_telephony_direction" VALUES('Совинтел_МГМН_старые_кмст','7485');
  sqlite> CREATE INDEX const_telephony_direction_complex_idx on const_telephony_direction(name,destcode);
  sqlite> COMMIT;
  sqlite>
  sqlite> pragma integrity_check;
  rowid 1 missing from index sqlite_autoindex_groups_1
  </pre>
  <pre>
  $ sqlite3 :memory:
  SQLite version 3.6.18
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite>
  sqlite> BEGIN TRANSACTION;
  sqlite> CREATE TABLE groups
     ...> (
     ...>         name TEXT collate NOCASE NOT NULL DEFAULT ''
     ...> );
  sqlite> INSERT INTO "groups" VALUES('Администраторы');
  sqlite> CREATE TABLE const_telephony_direction (
     ...>         name text not null,
     ...>         destcode text not null
     ...> );
  sqlite> INSERT INTO "const_telephony_direction" VALUES('Совинтел_МГМН_старые_кмст','7485');
  sqlite> CREATE INDEX const_telephony_direction_complex_idx on const_telephony_direction(destcode);
  sqlite> COMMIT;
  sqlite>
  sqlite> pragma integrity_check;
  ok
  </pre>

  So the problem is index support for redefined collation.

  <hr><i>drh added on 2009-09-16 12:10:09:</i><br>
  The original bug report and the three "anonymous" comments all originate from
  the same IP address, so presumably they come from the same person (hereafter
  the "OP").

  Apparently, the OP has compiled a custom version of the "sqlite3" command-line
  shell that contains a redefined NOCASE collation.  (See the URL in the
  2009-09-16 07:16:08 post for the code.)  However, this redefined NOCASE
  collating function contains one or more bugs which result in SQLite
  malfunctions.

  The OP is requesting help in debugging his custom, external NOCASE collation.

  Requests for help in debugging personal application should be directed
  to the SQLite users mailing list at sqlite-users@sqlite.org.  This ticketing
  system is reserved for tracking bugs associated with SQLite itself.


tkt_uuid: f1c2a689c64f2d51941be8fa0df84c7e7c6afd34
type: Build_Problem
status: Closed
subsystem: Other
priority: Immediate
severity: Cosmetic
foundin: 3.6.18
private_contact: 7c8d2d839a59e9e7114c498039fb0afd1f6ce120
resolution: Fixed
title: configure script ignores ``--with-tcl'' option when tclsh8.4 is installed.
comment:
  A configure script ignores ``--with-tcl'' option when tclsh8.4 is found in PATH.
  If a user want to use tclsh8.5 or later, he have to run "$ TCLSH_CMD=tclsh8.5 ./configure" to generate a Makefile.

  Because a configure script searches tclConfig.sh in each directory which is got 
  from "echo 'puts stdout $auto_path' | ${TCLSH_CMD}".

  It seems that "make test" requires Tcl 8.5 or later, so the setting priority to
  find path of Tcl library and tclsh should be described at "configure --help".

  <hr><i>drh added on 2011-10-13 18:33:53:</i><br>
  Fixed by [8118de2af33557f75b4f2f1b1194a21d46ccf7fe]


tkt_uuid: e25d9ea771febc9c311928c1c01c3163dcb26643
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: aed31b5dedc5990233487bd2f98644b1b9be949b
resolution: Fixed
title: Cascading delete trigger fails.
comment:
  <verbatim>
  PRAGMA recursive_triggers=1;
  CREATE TABLE node(
      id int not null primary key, 
      pid int not null default 0 references node,
      key varchar not null, 
      path varchar default '',
      unique(pid, key)
  );
  CREATE TRIGGER node_delete_referencing AFTER DELETE ON "node"
  BEGIN
    DELETE FROM "node" WHERE pid = old."id";
  END;
  INSERT INTO node(id, pid, key) VALUES(9, 0, 'test');
  INSERT INTO node(id, pid, key) VALUES(90, 9, 'test1');
  INSERT INTO node(id, pid, key) VALUES(900, 90, 'test2');
  DELETE FROM node WHERE id=9;
  SELECT * FROM node;
  </verbatim>

  result:
  <verbatim>
  900|90|test2|
  </verbatim>

  <hr><i>drh added on 2009-09-17 00:47:07:</i><br>
  The problem first introduced sometime after [dee1b8eb40] (2009-08-24), and on or before check-in [8a746fbfd5] (2009-08-31). The precise origin of the problem cannot be more precisely localized. This band of changes are where the trigger logic was refactored to support recurse triggers.

  Problem fixed by check-in [03e464be51a1]


tkt_uuid: 1ff6d29030e9757d052af0f394e56bea96b8b59b
type: Build_Problem
status: Closed
subsystem: Utilities
priority: Immediate
severity: Important
foundin: 2b2a1ef74e24857b8d18f6370894588fc2aa5ebb
private_contact: bab315d2cac51a9446a1315f82fe34aaf58992b5
resolution: Fixed
title: Compile fails with SQLITE_OMIT_TRIGGER
comment:
  Compile fails with SQLITE_OMIT_TRIGGER defined. In particular, these errors are reported:

    *  delete.c 466: ) expected

    *  insert.c 1255: Declaration terminated incorrectly in function sqlite3GenerateConstraintChecks

    *  insert.c 1257: Lvalue required in function sqlite3GenerateConstraintChecks

    *  insert.c 1365: Declaration terminated incorrectly in function sqlite3GenerateConstraintChecks

    *  insert.c 1368: Lvalue required in function sqlite3GenerateConstraintChecks

  All problems result from the fact that referenced symbols are #ifndef'ed out by SQLITE_OMIT_TRIGGER.

  <hr><i>dan added on 2009-09-21 16:08:20:</i><br>
  Fixed now. Thanks for the report.


tkt_uuid: b803a45d50378dc8b161995e3489d714b2acd26d
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
private_contact: 718c83cc9eb92cdd77f8fce1ae6fa7b90b433993
resolution: Rejected
title: Feature Request: MATCHES(VAL1%, VAL2, %VALn%, ...)
comment:
  I see that SQLite has already implemented a LIKE(a,b) function., so the name for this request can't be the ideal one.

  Just as the "WHERE COLUMN IN(VAL1, VAL3, VALn, ...)" function compares the value in COLUMN against the items in the IN(...) list, and returns TRUE if any of them are equal to it (a shorthand form of COLUMN=VAL1 OR COLUMN=VAL2 OR COLUMN=VAL3), I would like to have a function that does a standard LIKE comparison against a list of values:

  where COLUMN MATCHES('anthro%','paleo%','%morph%').

  This avoids having to repeat the column name over and over with many  OR operators:

  Where COLUMN LIKE('anthro%') OR COLUMN LIKE('paleo%') OR COLUMN LIKE('%morph%')
  ...

  Thanks for considering.


tkt_uuid: 15e11a2c8720f0506adee8d5c8cc28d90bf3eeee
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
private_contact: 718c83cc9eb92cdd77f8fce1ae6fa7b90b433993
resolution: Rejected
title: IfEmpty(a,b) : a function like IfNull(a,b) but for empty strings
comment:
  For use with data where the empty string has been entered into the column instead of NULL. 

  IfEmpty(a,b) would work analogous to IfNull(a,b) and return the first non-null, non-empty-string value, or return the empty string if both values are empty.

  Thanks for considering.

  <hr><i>anonymous added on 2009-09-18 12:06:49:</i><br>
  Great idea! Function coalesceEmpty(...) is helpful too. And operator "is Empty" may be added similar to "is null".

  There are some difficults when TCL interface store empty strings inside of NULLs. Empty string is useful in application but is not useful in SQL becouse there are not functions to handle these.

  P.S. I can add IfEmpty(a,b) and coalesceEmpty(...) functions or redefine internal by user extension. But I don't know how to create operator "is Empty". Please help!


tkt_uuid: af3450b7635ea231f7981ce1a8e7008a103a6247
type: Feature_Request
status: Closed
subsystem: Extensions
priority: Low
severity: Important
foundin: 3.6.17
private_contact: 67bbaf11deabcf0a4a461858ed27b5c541d05d4e
resolution: Rejected
title: Recursive virtual tables
comment:
  When I create virtual table where another virtual table in involved in xCreate procedures (e.g. echo(VTAB), where VTAB itself is virtual), 
  I'm getting error message: "SQL error: library routine called out of sequence" as result of sqlite3_declare_vtab() call.

  This happenes if VTAB was not used in current connection (i.e. if there is xConnection of one virtual table called within xCreate/xConnection of another virtual table). 
  If I previously call Select * from VTAB (kicks off xConnect), and then CREATE VIRTUAL TABLE VTAB2 USING echo(VTAB), I have no error.

  PS: echo module as implemented in test8.c must be slightly modified to support such recursive calls: echoDeclareVtab should use PRAGMA table_info() to obtain sql string and column names instead of select from sqlite_master.
  I can send source of modified echo module which can produce this bug upon e-mail request.


tkt_uuid: 56928bd084ea530eb8a0b3ebe5d2d9033fb1de7f
type: Build_Problem
status: Closed
subsystem: Utilities
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: ce08a27daf7f4e7a27c9ebb925d291a23225442e
resolution: Fixed
title: Cannot compile with SQLITE_OMIT_UTF16 and SQLITE_ENABLE_STAT2
comment:
  when compiled with SQLITE_OMIT_UTF16, in file utf.c, the routine sqlite3Utf8to16 is unavailable

  However, where.c uses it when SQLITE_ENABLE_STAT2 is defined in the routine whereRangeRegion

  <verbatim>
    char *zSample = sqlite3Utf8to16(  db, pColl->enc, aSample[i].u.z, aSample[i].nByte, &nSample  );
  </verbatim>

  <hr><i>anonymous added on 2009-09-19 07:25:42:</i><br>
  Ticket [1ff6d29030e9757d052af0f394e56bea96b8b59b] describes another SQLITE_OMIT problem in SQLite 3.6.18.

  <hr><i>dan added on 2009-09-21 16:34:44:</i><br>
  Fixed now. Thanks for the report.


tkt_uuid: 6f8b0c585550d8d7354b6a3e1207f98eb742746a
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Cosmetic
foundin: 3.6.18
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: Some minor genfkey fixups
comment:
  Since I wanted genfkey functionality and [http://www.sqlite.org/cvstrac/tktview?tn=3687] is unlikely to be addressed in the near future, I automatically extract the genfkey code from the shell.

  The detectSchemaProblem function and aSchemaTable should be declared static.

  <hr><i>shane added on 2009-10-21 13:50:42:</i><br>
  Check-in [c6f22f3e38].


tkt_uuid: 0bc616a2c815a1e945133a2aca705a786224d8cf
type: Build_Problem
status: Closed
subsystem: Other
priority: Immediate
severity: Critical
foundin: all
private_contact: 84dafd382563c2a5707df07f87aa123fe8d46ed7
resolution: Not_A_Bug
title: In wince 6.0, I can not compile sqlite.
comment:
  In wince 6.0, When I compile sqlite DLL, I Get some warnings and a error:

  sqlite3.c(11210) : warning C4013: 'localtime_s' undefined; assuming extern returning int

  sqlite3.obj : error LNK2019: unresolved external symbol localtime_s，referenced in function localtimeOffset

  <hr><i>shane added on 2009-09-22 13:32:09:</i><br>
  WINCE does not supply a "localtime" function.  You have a few options:

  1) Provide your own localtime_s() function.

  2) Add the compile option /Dlocaltime_s=gmtime_s to map calls from localtime() to gmtime() (timezone support for date/time functions will not be accurate)

  3) Omit localtime support all together with the compile option SQLITE_OMIT_LOCALTIME (you lose timezone support for date/time functions and a couple date/time formatting options)

  4) Omit all date/time support with the compile option SQLITE_OMIT_DATETIME_FUNCS

  5) Omit all floating point support (which also omits date/time support) with the compile option SQLITE_OMIT_FLOATING_POINT (this will require you to build from the full source distribution).


tkt_uuid: aad47726e98bbf4be9f476cd1a77436d922d89f6
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
private_contact: 3299242f9eeb048063fa0161cd817ed02be44d65
resolution: Rejected
title: Generator of sqlite3UpperToLower&#91;&#93; and sqlite3CtypeMap&#91;&#93; for single lang
comment:
  Is needed subj utility for any language.

  <hr><i>rogerb added on 2009-10-28 04:55:04:</i><br>
  Please explain clearly what it is you want on the sqlite-users mailing list.  Also point out why ICU or supplying your own collations is not sufficient.


tkt_uuid: c472df6c1cd16ee48c80c535a97362e802b38be2
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.18
private_contact: 3299242f9eeb048063fa0161cd817ed02be44d65
resolution: Duplicate
title: Segmentation fault by instead of insert triggers
comment:
  <pre>
  create table a(id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL);
  create view view_a as select * from a;
  CREATE TRIGGER view_a_insert instead of insert on view_a
  begin
    insert into a default values;
  end;
  insert into view_a(a) values (1);
  SQL error: cannot modify view_a because it is a view

  create table a(id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL);
  create view view_a as select * from a;
  CREATE TRIGGER view_a_insert instead of insert on view_a
  begin
    insert into a values (NEW.id);
  end;
  insert into view_a(id) values (1);
  Segmentation fault

  create table a(id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,a text);
  create view view_a as select * from a;
  CREATE TRIGGER view_a_insert instead of insert on view_a
  begin
    insert into a (a) values (NEW.a);
  end;
  insert into view_a(a) values (1);
  Segmentation fault
  </pre>

  <hr><i>drh added on 2009-09-21 12:57:42:</i><br>
  Duplicate of ticket [a696379c1f0886615541a48b35bd8181a80e88f8].
  Fixed by check-in [2b2a1ef74e24857b8d18f6370894588fc2aa5ebb].

  <hr><i>anonymous added on 2009-09-21 13:18:38:</i><br>
  Thanks! 3 lines of patch are fixed segfaults.

  But this is not duplicate:
  <pre>
  create table a(id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL);
  create view view_a as select * from a;
  CREATE TRIGGER view_a_insert instead of insert on view_a
  begin
    insert into a default values;
  end;
  insert into view_a(a) values (1);
  SQL error: cannot modify view_a because it is a view
  </pre>

  <hr><i>dan added on 2009-09-21 16:14:51:</i><br>
  See the "Syntax Restrictions On UPDATE, DELETE, and INSERT Statements Within Triggers" section of the [http://www.sqlite.org/lang_createtrigger.html|CREATE TRIGGER] documentation. The "INSERT INTO <tbl> DEFAULT VALUES" syntax is not
  allowed within a trigger definition. As a result, your INSTEAD OF trigger was never added to the database schema. Hence the "cannot modify view" message.


tkt_uuid: a8b1e82b8f93415ed4601be04469bcfee91ff43d
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.6.18
private_contact: 8e2d3673152fc451f2f1839dba7fe8f2162bedbd
resolution: Not_A_Bug
title: CAST failing for numeric
comment:
  While

  SELECT CAST(COALESCE(SUM(amount), 0) AS INTEGER

  returns as expected an integer, CAST doesn't work for NUMERIC:

  SELECT CAST(COALESCE(SUM(amount), 0) AS NUMERIC always returns float

  <hr><i>drh added on 2009-09-22 13:00:57:</i><br>
  FLOAT is NUMERIC.  Casting to NUMERIC simply converts strings and blobs into
  numeric values.  It does not change INTEGER into FLOAT.


tkt_uuid: 702bbe755b63e434ccc458b0635164379acd7805
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Severe
foundin: 3.5.9
private_contact: f301b1b3aa4bc49f917b95f936fb65054520acba
resolution: Overcome_By_Events
title: Invalid VDBE program built if ORDER BY used
comment:
  Hi,

  This is Peter Schaeffer from Progress/DataDirect. I ran a join with ORDER BY and got invalid results. The problem is the VDBE program built at runtime. The WHERE clause is implemented by comparing registers 5 and 3. However, register 3 is corrupted by a move.

  <hr><i>anonymous added on 2009-09-22 13:10:26:</i><br>
  Hi,

  This is Peter Schaeffer from Progress/DataDirect. I ran a join with ORDER BY and got invalid results. The problem is the VDBE program built at runtime. The WHERE clause is implemented by comparing registers 5 and 3. However, register 3 is corrupted by a move.

  I have a printout of the VDBE program showing the problem. Where should I send or upload the printout?

  Thank you

  Peter Schaeffer
  281-634-8340

  <hr><i>anonymous added on 2009-09-22 14:24:46:</i><br>
  I read some more code and obtained a general idea of what might be wrong. The WHERE clause has a simple column comparison (as in WHERE A.colx = B.coly). The expression evaluation code assumes that A.colx i cached in register 3. However, a Move much later inside the join loop overlays register 3.

  <hr><i>drh added on 2009-09-22 15:03:41:</i><br>
  The part of the code generator that is responsible for creating the VDBE
  code described here has been completely rewritten since version 3.5.9.
  This bug is probably fixed in later versions of SQLite.  

  If you will show us an example script that demonstrates the problem, we
  can verify that the problem has been fixed, and probably show exactly which
  check-in fixed the problem.  But without a reproducible test script, there
  is little we can do.

  <hr><i>anonymous added on 2009-09-22 19:47:26:</i><br>
  Will do.

  Thank you

  Peter Schaeffer

  <hr><i>anonymous added on 2009-09-23 04:12:15:</i><br>
  I reproduced the problem with a relatively simple program. How should I upload the program? 

  Thank you

  Peter Schaeffer

  <hr><i>drh added on 2009-09-23 13:42:57:</i><br>
  Yes, upload the program or email to the developers.

  <hr><i>anonymous added on 2009-09-23 15:41:14:</i><br>
  Where/how do/are files uploaded?

  What is the Email address for the developers?

  Thank you

  Peter Schaeffer

  <hr><i>drh added on 2009-09-23 15:52:46:</i><br>
  drh@sqlite.org

  <hr><i>anonymous added on 2009-09-24 16:48:55:</i><br>
  The test file has been sent.

  Thank you

  Peter Schaeffer


tkt_uuid: f51d9501800de5a0fb69d5048ce6662981b461ec
type: Compiler_Warning
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 38d192610b706551165d36818484340d04265463
resolution: External_Bug
title: memset warning with gcc 4.3.3
comment:
  While compiling latest version i get:

  In function ‘memset’,
      inlined from ‘exprDup’ at sqlite3.c:60527:
  /usr/include/bits/string3.h:82: warning: call to ‘__warn_memset_zero_len’ declared with attribute warning: memset used with constant zero length parameter; this could be due to transposed parameters

  It seems really wrong.. i checked and the error is also in fossil.

  Line is:

  <verbatim>
          memset(&zAlloc[nSize], 0, EXPR_FULLSIZE-nSize);
  </verbatim>

  <hr><i>drh added on 2009-09-23 11:54:18:</i><br>
  The 3rd parameter to memset is <i>not</i> a constant and is frequently
  non-zero, as you can easily verify by adding:

  <pre>
       assert( EXPR_FULLSIZE - nSize == 0 );
  </pre>

  To the line above the memset in question and seeing that the assertion 
  fires rather easily.

  This is a bug in GCC.  It appears to be a harmless bug in that correct
  code is still generated, but a bug nevertheless.

  <hr><i>anonymous added on 2010-01-11 14:13:28:</i><br>
  Opened GCC bug report here:
  [http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42688]


tkt_uuid: 4a03edc4c8c028c93e9269f64fc5e97f632c1166
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.6.18
resolution: Fixed
title: Index corruption following aggressive use of ON CONFLICT clause.
comment:
  The following SQL results in a database file that contains a corrupt
  index, as demonstrated by the "integrity_check" at the end:

  <verbatim>
      CREATE TABLE t1(
        a INTEGER PRIMARY KEY ON CONFLICT REPLACE,
        b UNIQUE ON CONFLICT FAIL
      );
      INSERT INTO t1 VALUES(1, 1);
      INSERT INTO t1 VALUES(2, 2);
      BEGIN;
        INSERT INTO t1 VALUES(1, 2);
      COMMIT;
      PRAGMA integrity_check;
  </verbatim>

  The index corruption can be fixed by running the REINDEX command.
  This bug is present in all versions of SQLite going back at least
  to version 3.1.3 on 2005-02-19. (Version 3.1.3 is the oldest version that is
  readily at hand for testing.)

  <hr><i>drh added on 2009-09-24 18:42:02:</i><br>
  Fixed in check-in [f0c56fa90dc95aff6fe6764b5ab75a90199247b0]


tkt_uuid: c9394a56c75ed37ac5048aa8adb413072fe54fd9
type: Support_Request
status: Closed
subsystem: Parser
priority: Immediate
severity: Important
foundin: 3.6.17
private_contact: 6411a4a22b6383e7173de128f65e2169f04197c7
resolution: Not_A_Bug
title: Sqlite reading all column data on selects.
comment:
  Hi, are there plans to stop sqlite3 from reading in all column data on specific select queries? i.e I would like sqlite to ONLY read column data for columns which are specified in the select and where clauses.

  Thanks.
  Adam.

  <hr><i>drh added on 2009-09-24 12:25:43:</i><br>
  This would be an excellent question to ask on the SQLite mailing list at
  sqlite-users@sqlite.org.  Please seek your answer there.


tkt_uuid: 3338b3fa19ac4abee6c475126a2e6d9d61f26ab1
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: f80625b2efab4ad1e319d657600a4ac7ced5d353
resolution: Fixed
title: ambiguous column name when using multiple JOIN USING clauses and WHERE clause
comment:
  <verbatim>
  $ sqlite3
  SQLite version 3.6.18
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE a (a INTEGER);
  sqlite> CREATE TABLE b (a INTEGER);
  sqlite> CREATE TABLE c (a INTEGER);
  sqlite> SELECT * FROM a JOIN b USING (a);
  sqlite> SELECT * FROM a JOIN b USING (a) JOIN c USING (a);
  sqlite> SELECT * FROM a JOIN b USING (a) WHERE a = 1;
  sqlite> SELECT * FROM a JOIN b USING (a) JOIN c USING (a) WHERE a = 1;
  SQL error: ambiguous column name: a
  sqlite> 
  </verbatim>

  This bug was found while working on a class table inheritance plugin for the Sequel Ruby ORM.

  <hr><i>drh added on 2011-10-13 14:44:54:</i><br>
  The column name is ambiguous.  Does it mean a.a or b.a?  The result is the
  same either way, but I don't expect the parser to know this.


tkt_uuid: f7e3d17a34c5254c80023b3ac6becd2e18843139
type: Support_Request
status: Closed
subsystem: Unknown
priority: Low
severity: Important
foundin: 3.5.9
private_contact: 2260ddd80d406400dcf2af8aa0f114c277123ee4
resolution: Not_A_Bug
title: Memory accumulate without commit
comment:
  Hi, all:
  I have a trouble to use SQLite3 without OUT OF MEMORY. 
  I compile SQLite3 with following options:
  SQLITE_ENABLE_MEMORY_MANAGEMENT
  SQLITE_DEFAULT_PAGE_SIZE=512
  SQLITE_MEMORY_SIZE=262144

  And I create a table with following SQL:
  CREATE TABLE plst_item(
      media_id    INTEGER,
      plst_id		INTEGER,
      idx			INTEGER		NOT NULL,
      PRIMARY KEY (media_id, plst_id))

  And then I have a loop to do following:

  BEGIN;
  for i = 1 to 100000
  {
      prepare SQL: INSERT INTO plst_item(media_id, plst_id, idx) values(?, ?, ?)
      sqlite3_bind_int64(stmt, 1, i);
      sqlite3_bind_int64(stmt, 2, i);
      sqlite3_bind_int64(stmt, 3, i);
      sqlite3_step(stmt);
      sqlite3_finalize(stmt);
  }
  COMMIT;

  The result will OUT OF MEMORY.
  But if I do commit in every 300 times insert, then there is no this issue.

  I found the memory requirement will be accumulated during INSERT.
  That means the memory will leak or used if we don't commit the transation.

  I think it is not meke scese, and hope someone can help me.

  <hr><i>anonymous added on 2009-09-25 01:26:26:</i><br>
  Hi, all: I have a trouble to use SQLite3 without OUT OF MEMORY. <p>

  I compile SQLite3 with following options: <p>
  SQLITE_ENABLE_MEMORY_MANAGEMENT <p>
  SQLITE_DEFAULT_PAGE_SIZE=512 <p>
  SQLITE_MEMORY_SIZE=262144 <p>

  And I create a table with following SQL: <p>
  CREATE TABLE plst_item( <p>
      media_id INTEGER, plst_id INTEGER, idx INTEGER NOT NULL, <p>
      PRIMARY KEY (media_id, plst_id))<p>
  <p>
  And then I have a loop to do following:<p>
  <p>
  BEGIN; <p>
  for i = 1 to 100000 <p>
  { <p>
  prepare SQL: INSERT INTO plst_item(media_id, plst_id, idx) values(?, ?, ?) <p>
  sqlite3_bind_int64(stmt, 1, i); <p>
  sqlite3_bind_int64(stmt, 2, i); <p>
  sqlite3_bind_int64(stmt, 3, i); <p>
  sqlite3_step(stmt); <p>
  sqlite3_finalize(stmt);<p>
  } <p>
  COMMIT;<p>
  <p>
  The result will OUT OF MEMORY. But if I do commit in every 300 times insert, then there is no this issue.<p>
  <p>
  I found the memory requirement will be accumulated during INSERT. That means the memory will leak or used if we don't commit the transation.<p>
  <p>
  I think it is not meke scese, and hope someone can help me.

  <hr><i>drh added on 2009-09-25 02:01:35:</i><br>
  This is a support request, not a bug report.  Please seek help using
  the SQLite mailing list at sqlite-users@sqlite.org.


tkt_uuid: 860d129c150caee25486eb6f781f4aa71f2564d7
type: Build_Problem
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.18
private_contact: f4471882e74bca357ca3cf7b8709a92897657c2a
resolution: Duplicate
title: builds with sqlite 3.6.18 depends on tchsh
comment:
  Attempting to build sqlite3 assumes tcl is installed.

  ./configure --prefix=/home/achowe/Projects/org/sqlite --disable-amalgamation --disable-tcl --without-tcl --enable-threadsafe --enable-debug --enable-static --disable-shared

  ...

  Installing threaded SQLite3...

  rm -f *.lo *.la *.o sqlite3 libsqlite3.la
  rm -f sqlite3.h opcodes.*
  rm -rf .libs .deps tsrc
  rm -f lemon lempar.c parse.* sqlite*.tar.gz
  rm -f mkkeywordhash keywordhash.h
  rm -f 
  rm -f *.da *.bb *.bbg gmon.out
  rm -f testfixture test.db
  rm -f common.tcl
  rm -f sqlite3.dll sqlite3.lib sqlite3.def
  rm -f sqlite3.c .target_source
  tclsh ./tool/mksqlite3h.tcl . >sqlite3.h
  /bin/sh: tclsh: not found

  <hr><i>anonymous added on 2009-09-25 08:05:18:</i><br>
  The build depends on TCL at compile time not at runtime.

  <hr><i>drh added on 2009-09-25 10:35:42:</i><br>
  Duplicate of ticket [7d96113ff0aac1d54c96f9c0268fad1b6d757892]


tkt_uuid: 58a75d62e08ed83d18cf0d3790ab61bd804acba2
type: Incident
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.17
resolution: Unable_To_Reproduce
title: encoding failure
comment:
  using the sqlite_exec() the callback returns the 5 char string "&#300;"  for an input string of 5 chars "&#300;"  if the input is a single wchar_t of value 300, the callback also returns 5 char string "&#300;"

  how can i tell the difference?
  2 different inputs, the same output :(

  <hr><i>drh added on 2009-09-26 15:14:42:</i><br>
  I am unable to reproduce this problem, possibly because I do not really
  understand what the problem is.  Please restate your problem on the
  sqlite-users mailing list ([mailto:sqlite-users@sqlite.org]) and we will
  review it there and reopen this ticket as necessary.  Thanks for using
  SQLite and thanks for reporting this problem.


tkt_uuid: d77ffcd7a5de30031286ad4f5c750fd3a3dcdb39
type: Code_Defect
status: Closed
subsystem: Shell
priority: Low
severity: Minor
foundin: sqlite-3.6.12-3.fc11.x86_64
private_contact: 453a6921018e965e98d9d5975a905575469c046e
resolution: External_Bug
title: Memory leak on failing INSERT statements on tables with UNIQUE INDEX
comment:
  ** Database

  <verbatim>
  CREATE TABLE leaktest ( recordid integer PRIMARY KEY AUTOINCREMENT, value varchar(32));
  CREATE UNIQUE INDEX leaktest_idx ON leaktest(value);
  </verbatim>

  SQL test query: INSERT INTO leaktest (value) VALUES ('value1')

  On the first insert, everything works fine and there are no memory leaks at all.
  On the second round a memory leak is happening.

  In the program I discovered this, I only get a 40 byte memory leak.  When using the sqlite3 command line utility, the leak is 68,424 bytes.

  ** Valgrind reports

  <verbatim>
  sqlite3 leaktest "INSERT INTO leaktest (value) VALUES ('value1')"
  ==10920== 68,424 bytes in 102 blocks are possibly lost in loss record 1 of 1
  ==10920==    at 0x4A0763E: malloc (vg_replace_malloc.c:207)
  ==10920==    by 0x35FEA1D271: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10920==    by 0x35FEA08D38: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10920==    by 0x35FEA08E17: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10920==    by 0x35FEA13934: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10920==    by 0x35FEA3C875: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10920==    by 0x4032A6: open_db (shell.c:1905)
  ==10920==    by 0x40705B: main (shell.c:3045)
  </verbatim>

  my own sqlite test program - sqlite leaktest "INSERT INTO leaktest (value) VALUES ('value1')"

  <verbatim>
  ==10936== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
  ==10936==    at 0x4A0763E: malloc (vg_replace_malloc.c:207)
  ==10936==    by 0x35FEA1D271: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10936==    by 0x35FEA08D38: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10936==    by 0x35FEA08E17: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==10936==    by 0x35FEA5A963: sqlite3_exec (in /usr/lib64/libsqlite3.so.0.8.6)
  ==10936==    by 0x401E58: sqlite_query (sqlite.c:261)
  ==10936==    by 0x402FA1: main (sqlite.c:697)
  </verbatim>

  Kind regards,
  David Sommerseth

  <hr><i>drh added on 2009-09-26 03:42:32:</i><br>
  Unable to reproduce the problem in either version 3.6.12 or in the tip.
  Attempted on Mac OSX only.  I'll boot up an x86_64 box tomorrow and try
  again there, but expect to see the same result.

  Please provide additional detail on how to reproduce the problem.

  <hr><i>anonymous added on 2009-09-26 09:31:07:</i><br>
  I have the test program in the source tree of my project.

  git clone git://eurephia.git.sourceforge.net/gitroot/eurephia/eurephia.git

  Unfortunately, this test program do not compile in the latest beta release which I have packaged.  I can send you tarball of the development tree if you want.

  With the source from git, I do the following:

  <verbatim>
  $ cd database/sqlite/
  $ gcc -o sqlite sqlite.c -lsqlite3 -I ../ -I ../../common/ ../../common \
  /eurephia_log.c ../../common/eurephia_nullsafe.c ../eurephiadb_mapping.c \
  ../../common/eurephia_xml.c `pkg-config libxml-2.0 --cflags --libs` \
  ../../common/passwd.c -DSQLITE_DEBUG ../../common/sha512.c ../../common \
  /eurephia_values.c ../../common/randstr.c \
  `pkg-config openssl --cflags --libs` -Wall -g -DHAVE_LIBXML2 -D_GNU_SOURCE
  $ sqlite3 leaktest
  SQLite version 3.6.12
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE leaktest ( recordid integer PRIMARY KEY AUTOINCREMENT, value varchar(32));
  sqlite> CREATE UNIQUE INDEX leaktest_idx ON leaktest(value);
  sqlite> .quit
  $ valgrind --leak-check=full ./sqlite leaktest "INSERT INTO leaktest (value) VALUES ('value1')"
  [...snip...]
  ==16929== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 2)
  ==16929== malloc/free: in use at exit: 0 bytes in 0 blocks.
  ==16929== malloc/free: 205 allocs, 205 frees, 115,218 bytes allocated.
  [...snip...]
  </verbatim>
  First run is good
  <verbatim>
  $ valgrind --leak-check=full ./sqlite leaktest "INSERT INTO leaktest (value) VALUES ('value1')"
  [...snip...]
  ==16953== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
  ==16953==    at 0x4A0763E: malloc (vg_replace_malloc.c:207)
  ==16953==    by 0x35FEA1D271: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==16953==    by 0x35FEA08D38: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==16953==    by 0x35FEA08E17: (within /usr/lib64/libsqlite3.so.0.8.6)
  ==16953==    by 0x35FEA5A963: sqlite3_exec (in /usr/lib64/libsqlite3.so.0.8.6)
  ==16953==    by 0x401E58: sqlite_query (sqlite.c:261)
  ==16953==    by 0x402FBB: main (sqlite.c:698)
  ==16953== 
  ==16953== LEAK SUMMARY:
  ==16953==    definitely lost: 40 bytes in 1 blocks.
  [...snip...]
  $
  </verbatim>

  Second run fails becuase of "SQL Error: column value is not unique", which leads to the memory leak.

  I'm running this test on a Fedora 11 x86_64, pretty much up-to-date.  Info about the installed sqlite package is here: [http://koji.fedoraproject.org/koji/buildinfo?buildID=97190]



  <hr><i>anonymous added on 2009-09-26 10:16:01:</i><br>
  I just ran the same reproducing routine on latest sources from the fossil repository, with almost the same findings.

  (sources are already checked out, current dir is root of the sqlite source dir)
  <verbatim>
  $ autoconf
  $ mkdir build
  $ cd build
  $ ./configure
  {...snip...}
  $ make
  {...snip...}
  $ .libs/lt-sqlite3 leaktest
  SQLite version 3.6.19
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE leaktest ( recordid integer PRIMARY KEY AUTOINCREMENT, value varchar(32));
  sqlite> CREATE UNIQUE INDEX leaktest_idx ON leaktest(value);
  sqlite> .quit
  $ valgrind  --leak-check=full .libs/lt-sqlite3 leaktest "INSERT INTO leaktest (value) VALUES ('value1')"
  ==23447== Memcheck, a memory error detector.
  ==23447== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
  ==23447== Using LibVEX rev 1884, a library for dynamic binary translation.
  ==23447== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
  ==23447== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
  ==23447== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
  ==23447== For more details, rerun with: -v
  ==23447== 
  ==23447== 
  ==23447== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 2)
  ==23447== malloc/free: in use at exit: 0 bytes in 0 blocks.
  ==23447== malloc/free: 173 allocs, 173 frees, 117,935 bytes allocated.
  ==23447== For counts of detected errors, rerun with: -v
  ==23447== All heap blocks were freed -- no leaks are possible.
  $ valgrind --leak-check=full .libs/lt-sqlite3 leaktest "INSERT INTO leaktest (value) VALUES ('value1')"
  ==23452== Memcheck, a memory error detector.
  ==23452== Copyright (C) 2002-2008, and GNU GPL'd, by Julian Seward et al.
  ==23452== Using LibVEX rev 1884, a library for dynamic binary translation.
  ==23452== Copyright (C) 2004-2008, and GNU GPL'd, by OpenWorks LLP.
  ==23452== Using valgrind-3.4.1, a dynamic binary instrumentation framework.
  ==23452== Copyright (C) 2000-2008, and GNU GPL'd, by Julian Seward et al.
  ==23452== For more details, rerun with: -v
  ==23452== 
  SQL error: column value is not unique
  ==23452== 
  ==23452== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 2)
  ==23452== malloc/free: in use at exit: 66,504 bytes in 75 blocks.
  ==23452== malloc/free: 173 allocs, 98 frees, 116,943 bytes allocated.
  ==23452== For counts of detected errors, rerun with: -v
  ==23452== searching for pointers to 75 not-freed blocks.
  ==23452== checked 129,384 bytes.
  ==23452== 
  ==23452== 66,504 bytes in 75 blocks are possibly lost in loss record 1 of 1
  ==23452==    at 0x4A0763E: malloc (vg_replace_malloc.c:207)
  ==23452==    by 0x4C3F4E1: sqlite3MemMalloc (sqlite3.c:12575)
  ==23452==    by 0x4C125B8: mallocWithAlarm (sqlite3.c:15877)
  ==23452==    by 0x4C12687: sqlite3Malloc (sqlite3.c:15905)
  ==23452==    by 0x4C1AC54: sqlite3MallocZero (sqlite3.c:16152)
  ==23452==    by 0x4C4522F: openDatabase (sqlite3.c:95699)
  ==23452==    by 0x403306: open_db (shell.c:1924)
  ==23452==    by 0x406B0B: main (shell.c:3072)
  ==23452== 
  ==23452== LEAK SUMMARY:
  ==23452==    definitely lost: 0 bytes in 0 blocks.
  ==23452==      possibly lost: 66,504 bytes in 75 blocks.
  ==23452==    still reachable: 0 bytes in 0 blocks.
  ==23452==         suppressed: 0 bytes in 0 blocks.
  </verbatim>

  Hopefully some better pointers to where to look inside the code :)


  <hr><i>anonymous added on 2009-09-26 12:05:28:</i><br>
  I managed to reproduce it with the latest fossil code on my own reproducer as well now, but now with more source code references.

  <verbatim>
  ==27836== 40 bytes in 1 blocks are definitely lost in loss record 1 of 1
  ==27836==    at 0x4A0763E: malloc (vg_replace_malloc.c:207)
  ==27836==    by 0x4C3F4E1: sqlite3MemMalloc (sqlite3.c:12575)
  ==27836==    by 0x4C125B8: mallocWithAlarm (sqlite3.c:15877)
  ==27836==    by 0x4C12687: sqlite3Malloc (sqlite3.c:15905)
  ==27836==    by 0x4C5B544: sqlite3_exec (sqlite3.c:74869)
  ==27836==    by 0x401E58: sqlite_query (sqlite.c:261)
  ==27836==    by 0x402F7C: main (sqlite.c:674)
  </verbatim>

  Which made me double check the SQLite3 docs ... and I'm very much embarrassed, but I had forgotten to call sqlite3_free() on the the error message.

  However, it still might be an issues with the sqlite3 binary.

  /me goes hiding under a rock


tkt_uuid: cff1a5293723efc00d7e52bbce5a733c4154bec7
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: sqlite-3.6.18
private_contact: 6a54797aeb0c9718a1026fb6109bb8fc7a232664
resolution: External_Bug
title: sqlite3_get_table() returns wrong result when using where clause
comment:
  (table row with Id = 1 exists, sqlite3_open() works fine)

  Sample query:
  "select * from WebLayout where Id = '1';"

  sqlite3_get_table() returns only one row with column names, but not the expected (second) row containing the values.

  sqlite3_exec() + callback delivers correct result as well as sqlite3 command line tool with same query.

  sqlite3_get_table() delivers correct results if no where clause is applied. (e.g. "select * from WebLayout;")

  Development evironment:
  Cygwin / GNU C++

  static link sqlite3.o

  <hr><i>drh added on 2009-09-26 14:20:52:</i><br>
  Please post to sqlite-users@sqlite.org for help in resolving your issue.
  Thx.


tkt_uuid: efeeb90f2ac169bbb8e12d59a3576dcec5aaac6c
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.5
resolution: Rejected
title: error messages should be per-thread not global
comment:
  See http://www.sqlite.org/cvstrac/tktview?tn=3498

  SQLite error messages are currently stored as a single instance on the connection handle.  The error message API returns an integer or char* of the most recent error in any thread.  Consequently if the connection is used in multiple threads concurrently then there is a race condition between an error happening in an API and retrieving the information.  (In the case of the char* it could now point to bogus memory and result in a crash.)

  The way this issue is handled in other libraries and operating systems is to have the error information be stored per thread.  For example errno on POSIX and GetLastError() on Windows both behave this way.

  The current documented workaround is to call sqlite3_mutex_enter(sqlite3_db_mutex(D)) before each SQLite api call, retrieve the error message/code if applicable and then release the mutex.  This adds overhead (at a guess 1% CPU), and requires every SQLite call to be "wrapped" by the caller.  It also requires the developers to have read the API page and understood the issue.  This issue should also be pointed out at http://www.sqlite.org/threadsafe.html

  Making the error code/string be per thread unilaterally will break any existing code that does things like an operation in one thread and retrieval of the error code/message in a second thread.  At a guess this kind of code is very rare.

  Examining real world code <a href="http://www.google.com/codesearch?q=sqlite3_errmsg">shows</a> that existing open source projects out there do not get this handling correct.  One example is the Dalvik virtual machine used in Android. (The issue is only a problem if a database connection is used by more than one thread which is harder to determine, but I believe Android does allow it.)

  Going forward I would say the current approach of requiring an extra layer of taking a mutex around calls AND error retrieval has not been followed and will result in "random" crashes for programs unless they do.  It also goes against the way error handling is normally done (cf errno and GetLastError()).  Looking at existing code, it would also be tricky to do on many codebases anyway.

  I suggest changing SQLite to behave such that thread local errors are used.  This will make existing code work "normally" again, and not require the extra layer of locking (although there is no harm if it is still present).  The codebases that will have problems are those doing API calls and error retrieval in different threads.  That can be handled by a #define making one of the behaviours default, or a per connection flag to say if there is one error stored or one per thread.


tkt_uuid: 62359d6511eba862ca2dbab1e7ea936dc1fb8f72
type: Feature_Request
status: Closed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.6.1
resolution: Rejected
title: xGetLastError should be dumped
comment:
  The xGetLastError function isn't useful.  In fact none of the SQLite codebase even calls it.

  If it is implemented it only works in trivial situations.  In most cases an error at the VFS layer causes lots of other VFS methods to be called (eg rolling back the journal).  Consequently any initial error will be wiped out by later operations.

  In theory you could enhance the method to have a list of operations (eg read/write/truncate/open/delete) and corresponding error codes, but that information is available by Process Monitor on Windows and strace on Linux.

  I suggest updating the documentation to say this function is obsolete and removing all existing unused implementations.  (Some VFS use it internally but they could instead just call errno/GetLastError instead of this extra layer of indirection and type mangling.)


tkt_uuid: 20f9d4fbbff3a368afe66c6460d77c5987366e2f
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.5
resolution: Fixed
title: sqlite3_overload_function ignores errors
comment:
  sqlite3_overload_function calls sqlite3CreateFunc but ignores any errors from it. For example if you call sqlite3_overload_function with a function name longer than 255 bytes then sqlite3CreateFunc will return an error but sqlite3_overload_function still returns SQLITE_OK.


tkt_uuid: 0b803bff856c644c135d849d1a308e023713ecc9
type: Code_Defect
status: Fixed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.6.2
resolution: Fixed
title: Return (error code) of xCurrentTime is always ignored
comment:
  xCurrentTime is defined to return an error code. However no SQLite code ever looks at the return code and consequently ignores any errors.

  xCurrentTime is used in two places. One is for profile where failure doesn't matter. The other is for date/time functions. If xCurrentTime returns an error, then this should too:
  <pre>
     select date('now')
  </pre>
  Looking at the supplied vfs, the unixCurrentTime imlementation is a little sloppy as it ignores any errors from getttimeofday() which can fail if you don't have permissions to read the time.

  Also for some strange reason xCurrentTime is documented as returning zero on success and one on error. Why not just have it return a regular SQLite error code?

  Windows CE code does check for error. Normal Windows doesn't although no possible errors are documented. OS/2 does return a status but no possible errors are documented


tkt_uuid: 7d32c69b50f89dcbe21a7d0742379eeb0e92fec1
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Cosmetic
foundin: 3.6.0
resolution: Fixed
title: sqlite3LoadExtension truncates file names
comment:
  The effect of this "bug" is that if an extension fails to load from a filename longer than around 270 characters then the resulting error message (which is likely to be shown to a user of a SQLite based program) will have the filename truncated.  A brief search shows people having errors with extension loading but I didn't find any instances where the filename could not be determined (yet).

  Look at the code for sqlite3LoadExtension where it handles errors from sqlite3OsDlOpen as well as the next error block down.  In both cases an arbitrary size of 300 bytes is chosen to include both the error text as well as the filename.  There is also a stack allocation for the text followed by a heap allocation via a strdup equivalent.

  300 bytes is cutting it very short for just the filename.  Throw in drive letters, prefixes like "/usr/local/lib/" or "\Program Files", nice long application names, subdirectories to support files, some Unicode, version numbers etc and it is very easy to overflow.

  A simpler solution is to use sqlite3_mprintf with the default message plus the filename (and ideally some slack/padding).  That also avoids the stack allocation.  

  (The slack/padding is a bit of a pain but some sort of %* s format to make spaces and then putting in a null after the last non-space character would work.)


tkt_uuid: eb620916be52995fd1ee03dd61d2ae04848570b6
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Shell doesn't print header if multiple statements on one line
comment:
  If you have headers turned on and have multiple statements on one line then headers are only printed for the first statement.

  <pre>
  sqlite> create table foo(x); create table bar(y); insert into foo values(1); insert into bar values(2);
  sqlite> .header on
  sqlite> select * from foo ; select * from bar;
  <b>x
  1
  2</b>
  </pre>

  The output should have included <tt>y</tt> before the row from bar (value 2).

  <hr><i>rogerb added on 2009-09-29 04:17:25:</i><br>
  The same issue happens when echo is on.  The entire line is printed rather than each statement as it is executed:

  <pre>
  sqlite> .echo on
  sqlite> select 3; select 4;
  select 3; select 4;
  3
  4
  </pre>

  I would expect:

  <pre>
  sqlite> .echo on
  sqlite> select 3; select 4;
  select 3;
  3
   select 4;
  4
  </pre>

  <hr><i>rogerb added on 2009-10-19 05:31:41:</i><br>
  Also [http://www.sqlite.org/cvstrac/tktview?tn=2919]

  <hr><i>shane added on 2009-10-22 21:26:03:</i><br>
  Check-in [790402c150].

  <hr><i>shane added on 2010-01-18 18:16:31:</i><br>
  Additional changes in check-in [7080ae3bc3].


tkt_uuid: f12a9eeedc9465842636ec617693a0c0e264c04c
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Cosmetic
foundin: 3.6.18
resolution: Fixed
title: Shell command argument parsing is lax
comment:
  The shell commands mostly don't complain about extra arguments.  Some examples that should really generate errors but don't:

  <pre>
  sqlite> .header the a team is back
  sqlite> .backup main foo surplus arguments
  sqlite> .bail says this takes on or off
  sqlite> .databases doesn't care either
  sqlite> .dump does not care if the tables do not exist or even are the number 3
  sqlite> .echo are the ghostbusters out there?
  sqlite> .exit with code 22
  </pre>

  (I got bored having illustrated the point :-)

  <hr><i>rogerb added on 2009-10-01 02:04:07:</i><br>
  Another example is in parsing the .mode argument.  You can supply any substring of a valid mode which doesn't appear to be the intention of the code (why check for "line" and "lines", "column" or "columns") or the error message.

  <pre>
  sqlite> .mode ddd
  mode should be one of: column csv html insert line list tabs tcl
  sqlite> .mode col
  sqlite> .mode c
  sqlite> .mode ht
  sqlite> .mode t
  </pre>

  The actual mode that gets selected is based on the first match in the C code (eg "line" is checked for before "list" and "tcl" is checked before "tabs").

  Suggested code fix is to change the strncmps to a simple strcmp

  <hr><i>shane added on 2009-11-06 17:22:36:</i><br>
  Updated shell.c to improve "dot" command argument handling.  Added some basic tests to shell1.test to verify this.  

  Check-in [9fb6991933].   


tkt_uuid: dc3a6e5b31378eea92795c64ed25385eb07d61ed
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Shell doesn't htmlify column headers
comment:
  <pre>
  sqlite> .mode html
  sqlite> .header ON
  sqlite> select 'P&L';
  &lt;TR&gt;&lt;TH&gt;'P&amp;L'&lt;/TH&gt;&lt;/TR&gt;
  &lt;TR&gt;&lt;TD&gt;P&amp;amp;L&lt;/TD&gt;
  &lt;/TR&gt;
  </pre>

  The fix is that <tt>output_html_string</tt> should be called when outputting the header row just like it is for the rows.

  <hr><i>shane added on 2009-10-21 14:09:03:</i><br>
  Check-in [c8815fcd88].


tkt_uuid: 668fe2263793beea87df571d646a0b8be2ecc4dc
type: Feature_Request
status: Closed
subsystem: Parser
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Works_As_Designed
title: pragma table_info doesn't accept database names
comment:
  <pre>
  sqlite> pragma table_info(main.foox);
  SQLError: near ".": syntax error
  </pre>

  It would be nice if the database name was accepted since table_info is the only reasonable way to find out information about columns short of parsing the SQL.

  This really only matters if you are using attached databases.

  <hr><i>ralfjunker added on 2009-09-29 11:58:10:</i><br>
  According to the [http://www.sqlite.org/pragma.html|PRAGMA documentation], database names must precede the PRAGMA keyword, not the pragma-value.

  <verbatim>PRAGMA main.table_info(foox);</verbatim>


  <hr><i>rogerb added on 2009-09-29 18:26:41:</i><br>
  My mistake.  It is a little illogical though.


tkt_uuid: 35c54c8749872e4165328d6f12acbcd0930fd9a1
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Some os win symbols should be declared static
comment:
  Some the Windows vfs functions should be declared static but aren't.  Examples are winDlSym, winDlClose and winCurrentTime.

  (Having them public generates bogus symbols for tools that automatically generate export tables.)

  From http://www.sqlite.org/cvstrac/tktview?tn=3839


tkt_uuid: e123bd2242e87b88e252a002f1503b36d6f90395
type: Feature_Request
status: Closed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.3.7
resolution: Rejected
title: File handles aren't always immediately released on sqlite3_close
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=1990]

  Due to "quirks" in POSIX locking, file handle closing may be deferred until later which can give the appearance of them leaking.  The original ticket gives lots more detail.

  <hr><i>drh added on 2009-11-16 12:33:19:</i><br>
  This is not a defect and won't be fixed.  Applications can use the "unix-none"
  VFS if they do not want to make use of posix advisory locking.


tkt_uuid: e063139eb3f8df45a2dbba89fe968f6de0bba31d
type: Build_Problem
status: Fixed
subsystem: Makefile
priority: Immediate
severity: Important
foundin: 3.5.1
resolution: Fixed
title: Download .zip excludes shell but .tar includes it
comment:
  The shell source (shell.c) is included in the tar distribution but is excluded from the zip distribution.  It should also be included in the zip distribution.  This causes problems for Windows developers as they typically can only extract zips and not .tar.gz.  The shell is trivial to compile on Windows too (<tt>COMPILER -o sqlite.exe shell.c sqlite3.c</tt>) so this is actually useful.

  See also [http://www.sqlite.org/cvstrac/tktview?tn=2761]

  <hr><i>rogerb added on 2009-09-30 09:01:49:</i><br>
  The .def file is also requested.

  <hr><i>anonymous added on 2010-03-31 10:35:58:</i><br>
  It looks like this ticket is the continuation of the old cvstrac ticket [http://www.sqlite.org/cvstrac/tktview?tn=2761]

  But in this ticket, the description is not very clear about the actual problem. It's not so much that the shell.c is missing from the .zip and present in the .tar.gz rather than the fact that shell.c is missing from the amalgamation distribution (be it .zip or .tar.gz).

  I think shell.c should get included into the amalgamation distribution because this is what is documented on [http://www.sqlite.org/howtocompile.html]


tkt_uuid: 72adc99de9f6a443943557043ad8d47da1f76063
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: shell mode insert doesn't output blobs as blobs
comment:
  In mode insert the shell should show blobs using the X'' notation (.dump does this correctly).  Instead it just spews out the raw bytes.

  <pre>
  sqlite> .mode insert
  sqlite> select X'4445464748';
  INSERT INTO table VALUES('DEFGH');
  </pre>

  <hr><i>shane added on 2009-10-22 17:30:20:</i><br>
  Check-in [a2ad9e6363].

  <hr><i>shane added on 2009-10-22 18:15:07:</i><br>
  Also check-in [ab99faca6c].

  <hr><i>shane added on 2009-10-22 21:27:19:</i><br>
  Also check-in [790402c150].


tkt_uuid: 19b44e35753baae47ad997559f4dc333441930c6
type: Feature_Request
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Important
foundin: 3.6.18
resolution: Fixed
title: sqlite3_bind_text inconsistent with NULL pointers, zero length etc
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=4040]

  The OP is having problems because a sqlite3_bind_text with parameters of (NULL, 0) actually appears to bind null instead.  H13533 is somewhat woolly about this.

  On examining the code the ultimate cause is clear in bindText (also called to bind a blob).  The binding is first cleared (ie set to null).

  If the pointer is NULL then nothing else is done, irrespective of the length passed in.  The interface requirements do not cover this situation.

  I suggest that if the pointer is NULL then SQLITE_MISUSE is returned no matter what the length passed in is.  (This is obviously a bug in the application.  SQLite would be well within its rights to crash.)

  Trying to do clever things (eg ignoring pointer if length is 0) leads to more complications like what the destructor should be etc.


tkt_uuid: 8440a8e22fb4627349ca3edb49da2fbfbd5a3619
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.4.2
resolution: Not_A_Bug
title: LENGTH & QUOTE stop at first zero
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2613] (Also discusses some other stuff)

  The implementation of the length builtin function (lengthFunc) stops at the first zero character.  It should be using sqlite3_value_bytes to get the length instead. Same issue with quote/quoteFunc. In practise this breaks strings with embedded zeroes.

  <pre>
  sqlite> select length(x'444546004748');
  6
  sqlite> select length(cast(x'444546004748' as char));
  3
  sqlite> select quote(cast(x'444546004748' as char));
  'DEF'
  </pre>

  <hr><i>drh added on 2009-10-20 15:31:29:</i><br>
  I disagree.  I think that a string ends at the first nul character and that
  length() and quote() should both stop processing at that point.

  <hr><i>rogerb added on 2009-10-20 18:13:01:</i><br>
  You didn't give any reasoning :-)  In any event it should be pointed out that C strings are defined as null terminated, not SQL strings.  I like that SQLite treats strings correctly not making zero be special.  When I bind text or get it returned and say how long it is then SQLite pays attention to me.  LENGTH and QUOTE are the exceptions and inconsistent.


tkt_uuid: 3211d778c0e445cc762a372dfc7d8acaf372cecf
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Cosmetic
foundin: 3.4.2
resolution: Rejected
title: replace function works on blobs
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2613] (plus other discussion)

  The replace documentation states that it works on strings.  However it also accepts blobs for any of the parameters and coerces the the result to a string.

  It should really error when provided with blobs.

  If blobs are going to be supported then it should be required that all parameters are either string or blobs (ie all 3 the same type) and the output type be the same as the parameters type. cast can be used on the parameters to clarify if this is a byte based or character based operation.

  The code is fairly complicated already.  Being able to mix and match bytes with characters is likely to lead to bugs related to encodings (eg different results depending on utf16 or or utf8) and the possibility of producing invalidly encoded result strings by replacing the wrong bytes.

  <hr><i>drh added on 2009-11-16 12:36:30:</i><br>
  I think REPLACE() works fine just as it is.

  <hr><i>rogerb added on 2009-11-16 19:18:34:</i><br>
  Are you absolutely sure there is no combination of parameter types and data that will not lead to invalid UTF8/16 and consequent buffer overruns?

  It is also an application error to be mixing bytes/blobs with text.

  <hr><i>drh added on 2009-11-16 20:59:45:</i><br>
  One is never 100% sure of anything with software of this complexity.
  However, we do make a point of testing invalid UTF8 and UTF16 byte sequences
  to make sure they do not overflow any buffers. You can view a few such
  tests at [http://www.sqlite.org/src/artifact/8084dad398].  There are many
  more in [http://www.sqlite.org/th3.html | TH3].  Running such tests with
  valgrind gives us confidence that the buffers never overflow.

  At one point, we allowed read overruns of up to 2 bytes on invalid UTF, 
  under the assumption that such overruns were harmless.  But that assumption
  was proved wrong (see http://www.sqlite.org/src/info/3fe897352e) and such
  overruns have now been fixed.


tkt_uuid: 43d2aa29b9cc8e8c9e0d67c03c6879112cf75d59
type: Documentation
status: Closed
subsystem: Documentation
priority: Immediate
severity: Important
foundin: 3.6.16
resolution: Overcome_By_Events
title: H13015 needs to be completed
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3984]

  The requirements should be answers not questions :-) The prepare documentation hints at the answer which is that pzTail ends up pointing to the null terminating the string.


tkt_uuid: f5cb008a650a922180ab80d8b340fc0cbe0fb8d8
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Shell silently ignores extra parameters
comment:
  <pre>
  > sqlite3 ":memorydb:" "select 3" "select 4"
  3
  </pre>

  Cosmetically related the usage message says the parameter is SQL when dot commands are supported too (eg .dump).

  <hr><i>shane added on 2009-11-11 04:17:15:</i><br>
  Fixed and added some test cases to very basic handling of command line options.

  Check-in [09b4f19f10].


tkt_uuid: 2e047cbefa54099ea6866d7b7e68857aa25d1eab
type: Performance_Issue
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Critical
foundin: 3.6.5
resolution: Overcome_By_Events
title: select DISTINCT performance could be better
comment:
  See [http://www.sqlite.org/cvstrac/tktview?tn=3979] which includes benchmarks against MySQL, profiling information and TCL code to repeat the test.

  <hr><i>anonymous claiming to be jason.freund@access-company.com added on 2010-02-24 19:55:35:</i><br>
  I'm not sure if this is related, but when we tried upgrading from 3.4.0 to 3.6.5, most of our regression tests got much slower.

  A comment from Dr Hipp suggests it may be a regression introduced in 3.5.3 due to a change in select DISTINCT.

  See: http://www.mail-archive.com/sqlite-users@sqlite.org/msg39321.html


tkt_uuid: bd770b2c529cbf56cca2302d5a5cae3b6d3b37f0
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Important
foundin: 3.6.18
resolution: Fixed
title: shell doesn't return error exit code on import errors
comment:
  See also [http://www.sqlite.org/cvstrac/tktview?tn=3966]

  <pre>
  $ ./sqlite3 ':memory:' '.import notexist notexist'
  Error: no such table: notexist
  $ echo $?
  0
  $ sqlite3 ':memory:' 'select bad'
  SQL error: no such column: bad
  $ echo $?
  1
  </pre>

  <hr><i>shane added on 2009-10-23 00:37:10:</i><br>
  Check-in [009efad0f4].


tkt_uuid: 8f81455baf1099c16b0da49c1e9c6c735e2e94e3
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.6.21
resolution: Not_A_Bug
title: fts3GetVarint32 sets random number on error
comment:
  Original at [http://www.sqlite.org/cvstrac/tktview?tn=4035]

  fts3GetVarint32 has a 64 bit local variable that is uninitialized and then passed by address to fts3GetVarint.  (Some compilers whine about this.) If fts3GetVarint has an error it does not fill in the variable but fts3GetVarint32 does with the random stack value.

  Looking through the rest of the code it is extraordinarily rare for an error return of fts3GetVarint(32) to be looked for.

  fts3GetVarint32 should not fill in value on getting zero from fts3GetVarint.  Or both routines could be made consistent and set pointed to value to zero on error.

  <hr><i>anonymous added on 2009-12-10 17:09:21:</i><br>
  In addition fts3GetDeltaVarint also calls sqlite3fts3GetVarint, with same issue/potential bug.


tkt_uuid: 6da68f691b0716379089a169951a1e25f53709df
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Cosmetic
foundin: 3.6.18
resolution: Fixed
title: tables says you provide a like pattern but it adds % in code
comment:
  The tables help says:

  <pre>
  .tables ?PATTERN?      List names of tables matching a LIKE pattern
  </pre>

  However the code does this:

  <pre>
  "WHERE type IN ('table','view') AND name LIKE '%'||shellstatic()||'%' "
  </pre>

  This means the wildcard % is added on either side of what the user supplied.  The help should either be corrected to say "List names of tables with PATTERN in their name" (matches the code) or the code shouldn't be adding % (ie match the doc).

  <hr><i>rogerb added on 2009-10-05 04:16:17:</i><br>
  dump treats the table names as a like pattern but doesn't mention it in the help/usage

  <hr><i>shane added on 2009-10-24 02:00:11:</i><br>
  Check-in [7201244e67].


tkt_uuid: 1a1790ca1a9f1438f0c2e2dce57a68139efbdcf2
type: Feature_Request
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Important
foundin: 3.6.18
resolution: Rejected
title: No way to signal an error in collation callback
comment:
  (From discussion on mailing list and prior experience)

  If your user defined collation routine has an error then there is no way to tell SQLite that is the case.  (It is possible for user defined functions to signal errors.)

  At the very least the documentation should be updated saying what to do.  (Calling sqlite3_interrupt would seem to be the best).  Ideally a more pertinent mechanism should be provided.


tkt_uuid: 89668ca167638c94e75ed9892f9cb0bcf36c44a4
type: Feature_Request
status: Fixed
subsystem: Shell
priority: Low
severity: Cosmetic
foundin: 3.6.18
resolution: Fixed
title: shell can support .timer on Windows
comment:
  The shell doesn't currently support the .timer command on Windows.  On POSIX getrusage is used.  On Windows the same information can be found by using GetProcessTimes documented at [http://msdn.microsoft.com/en-us/library/ms683223(VS.85).aspx].  Use GetCurrentProcess() for the process id.  The filetime parameters must all be provided and are disguised 64 bit quantities in 100 nanosecond resolution.  There are several examples of using this function on the web.

  <hr><i>shane added on 2009-10-21 03:39:47:</i><br>
  Check-in [83216fbe90].


tkt_uuid: 4776dee07aa8026fa2eaeed947bd65e2e5f545ec
type: Documentation
status: Closed
subsystem: Documentation
priority: Immediate
severity: Minor
foundin: 3.6.14.2
resolution: Overcome_By_Events
title: Documentation and code about registering functions
comment:
  There are some issues mainly about mismatches/omissions between documentation, requirements and the code.

  The relevant documentation is at [http://www.sqlite.org/c3ref/create_function.html]

  "Any attempt to create a function with a longer name will result in SQLITE_ERROR being returned."

  Actually SQLITE_MISUSE is returned.

  From [http://www.sqlite.org/cvstrac/tktview?tn=3886]

  It says that the first time you call create_function it overrides the builtin function of the same name.  Subsequent calls only override if nArg is the same.  This (overriding builtins) is not mentioned in any requirements specs.  The specs and documentation should also mention that if you delete all user defined variants of a function then the builtin will "resurface".  Also that you can't delete the builtins (at least that is how the code in sqlite3FindFunction appears to work always searching builtins if matchquality is zero for user defined functions).


tkt_uuid: 0322cd49c0f844a22d34c687a608353bfef90d9d
type: Feature_Request
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Important
foundin: 3.5.9
resolution: Fixed
title: create_function_v2 that takes destructor
comment:
  I would be overjoyed if you could add create_function_v2 that takes a destructor callback in the same spirit as create_collation_v2 and create_module_v2.

  It is currently quite difficult to tell when a registered function is no longer needed.  (The rules involve case conversion, number of arguments, preferred encodings etc.)  In some cases the user code is several steps removed from SQLite (language bindings, abstractions etc) so it isn't obvious.  In my Python wrapper the only safe thing I could do is never garbage collect any registered functions until the connection is closed.

  [http://www.sqlite.org/cvstrac/tktview?tn=3141]

  <hr><i>rogerb added on 2009-10-05 18:18:36:</i><br>
  Another reason for needing this is if I make my code a loadable extension then it isn't my code that closes the connection so I don't know when the connection is closed and hence can free the functions.

  A nasty hacky workaround is to register a collation with a long unusable name and detect when that is being deleted and then delete the functions.

  <hr><i>rogerb added on 2010-10-11 04:23:46:</i><br>
  This has been implemented in 3.7.3.


tkt_uuid: ba7fdb568d6c80b90dfdc66d44e8a6f29a9ebd0f
type: Feature_Request
status: Fixed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Page size should auto default to file system block size
comment:
  See [http://www.sqlite.org/cvstrac/tktview?tn=2931] which detects the sector size (not the same as page size or filesystem block size!)

  With the hard drive industry wanting to move to 4,096 byte sectors and SSD drives preferring operations that match their erase block size it is becoming increasingly important that SQLite defaults match the hardware for performance as well as reliability (erase blocks).

  On POSIX (f)statvfs is useful.  On Windows GetDiskFreeSpace is useful (cluster size matches page size well).

  <hr><i>rogerb added on 2009-10-12 04:24:32:</i><br>
  See also [http://www.sqlite.org/cvstrac/tktview?tn=3452] although it confuses sector size with page size.

  In any event the VFS reports on sector size (the minimum unit of work the disk can do atomically).

  Probably the easiest way of adding this functionality is to allow "pragma page_size=0" to mean "please set it to a reasonable default" and using fstatvfs under posix to get a reasonable value.

  Ticket 3452 does mention how ZFS gets especially bad performance with the default SQLite page size.


tkt_uuid: 1c67bd6ce9d7fd366fdc92123af91da0750256b7
type: Feature_Request
status: Closed
subsystem: Shell
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Rejected
title: Shell Control-C (break) handling could be better
comment:
  If you press ctrl-C while entering SQL, the shell does interrupt the database under the hood (good) but still leaves you entering SQL.  It should abort the entry of the SQL and return to the primary prompt:

  <pre>
  sqlite> select
     ...> ^C^C^C^C^C
     ...> stop this ^C^C^C^C^C^C
  </pre>

  What I would expect:

  <pre>
  sqlite> select
      ..> ^C
  sqlite> 
  </pre>


tkt_uuid: 2466653295e655ce00990eae58939c0d33ed0b5c
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: Various dump issues
comment:
  While the dump command is running the database is not locked (eg BEGIN IMMEDIATE).  It is possible (but unlikely) for another process to mess with the database while the dump is running (very small windows of time).  Worst case another process could start a transaction causing the dump to timeout mid-dump.

  There is a zErrMsg variable but it isn't passed to any of the dumping helper functions nor is the error response of those functions checked.  This means that if any errors are encountered they are silently ignored.

  If SQLITE_CORRUPT is returned then error recovery repeats the effort to dump the table by reverse rowid (ie most recent writes first).  Again this appears to be silent.

  In the version after 3.6.18 PRAGMA foreign_keys is turned off at the beginning of the dump (even if no foreign keys are present) and never turned back on.

  For the record when searches work in Fossil I was concerned that virtual tables are detected by looking for exactly "CREATE VIRTUAL TABLE" but testing shows that if the table is created using lower case, spaces and tabs etc they are all normalized to upper case single space separated before being stored in sqlite_master so there is no problem.

  <hr><i>shane added on 2009-10-26 01:35:03:</i><br>
  For .dump, if you want to lock the database during the dump, you should use:

  <pre>
  sqlite3> BEGIN IMMEDIATE;
  sqlite3> .dump
  sqlite3> ROLLBACK;
  </pre>

  I'll take a look at the other issues.

  PS.  Not sure if you were pointing out an issue with the "CREATE VIRTUAL TABLE" comment or not.  



  <hr><i>rogerb added on 2009-10-26 01:55:04:</i><br>
  The problem is the dump code itself does not lock the database.  It expects the database to not change while it is working so if someone else was changing it the .dump code would (silently) keep plodding on.  The dump_callback function goes off and issues other queries as an example although it looks like the database would be reader locked while that happens so probably not an issue.

  The other comment was because on examining the dump source you can see string comparisons against text like "CREATE VIRTUAL TABLE" and my first thought was well what if it was created as "CrEaTe     VirTUAl  table".  The answer is the SQLite core uses the fixed uppercase strings and so it isn't a problem.


tkt_uuid: f9af981dd2a202009ae250c48bca968eb8d1c7d0
type: Documentation
status: Fixed
subsystem: Documentation
priority: Immediate
severity: Important
foundin: 3.6.18
resolution: Fixed
title: sqlite3_shutdown documentation needs to mention danger
comment:
  The documentation for sqlite3_shutdown doesn't actually mention what bad things will happen if you call it while statements are executing.  The following changes are suggested to the documentation:

    *  A first paragraph saying that if you are using a conventional operating system directly supported by SQLite (eg Mac, Linux/Unix, Windows) without any SQLite customizations then there is no need to call these functions.

    *  A second paragraph explaining that sqlite3_shutdown must not be called unless *all* connections have been closed.  No SQLite object must be in use in any way.

  This report is because someone was wondering about calling sqlite3_shutdown in order to abort execution if there is an error in collation callback (ie a possible workaround for [1a1790ca1a9f1438f0c2e2dce57a68139efbdcf2]).  Unless you are familiar with SQLite internals on reading the documentation it didn't seem like that bad an idea...

  <hr><i>drh added on 2009-10-20 15:28:27:</i><br>
  Check-in [b2aa48b52f1]


tkt_uuid: 94c04eaadb5382b3859a50b94c5586e849bd92af
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.6.18
resolution: Fixed
title: Crash in asyncRead() with large (>2GB) database file
comment:
  I've encountered segmentation fault at sqlite3async.c:715. The problem
  is on sqlite3async.c:712:

         nCopy = (int)MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);

  My actual numbers: iAmt = 16, iBeginOut = 2147844072. After
  subtraction we're getting -2147844056 or FFFFFFFF7FFA8028 in hex which
  after truncating to int gives 7FFA8028, a positive number. And it
  blows away the next check if( nCopy>0 ).


tkt_uuid: 8550ecca708e42de9a81dc06ecd67bd25359349a
type: Code_Defect
status: Fixed
subsystem: VFS
priority: Immediate
severity: Severe
foundin: 8382867956
private_contact: 22d2ebb9a6dbfa47bd4b558ba855d0331d912298
resolution: Fixed
title: findLockInfo uses memcmp on non-zeroed, = assigned struct
comment:
  findLockInfo() in os_unix.c creates the pLock structure via malloc but doesn't memset 0 the struct, the pLock->lockKey is set via = assignment and is later compared/looked up via memcmp against other lockKey, the llvm gcc 4.2 compiler doesn't copy the holes in the struct and so lockKeys can fail to compare correctly if the values in the holes happen to differ.  1-line fix: <br>
  <code><br>
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;goto exit_findlockinfo;<br>
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}<br>
  +&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;memset(&pLock->lockKey, 0, sizeof(lockKey));<br>
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;pLock->lockKey = lockKey;<br>
  &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;pLock->nRef = 1;<br>
  </code>

  <hr><i>drh added on 2009-10-07 23:43:40:</i><br>
  Fixed by check-in [56f609da4bfcc794e1f8573652dcde8515e87098]


tkt_uuid: ebde3f66fc64e21e61ef2854ed1a36dfff884a2f
type: Feature_Request
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Minor
foundin: 3.3.7
resolution: Works_As_Designed
title: Shared cache has different busy behaviour than non-shared
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2010]

  The shared cache immediately returns SQLITE_LOCKED instead of invoking the busy handler.  There is an expectation that turning shared cache on or off would have no effect on existing code - ie it should neither need to know or care if shared cache is in use.  The busy handling is a big difference.


tkt_uuid: 23b212820161c6599cbf414aa99bf8a5bfa5e7a3
type: Feature_Request
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Minor
foundin: 3.3.2
resolution: Fixed
title: Uninformative error message on check constraint failure
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=1648]

  If a check constraint fails then all you get is "constraint failed".  Even if you name your constraints, have more than one etc you don't get any useful information in the error message such as the constraint name or columns.  The original ticket included a poster who had a partial patch to fix this.

  <pre>
  sqlite> create table emp(id text unique, sex text constraint chk_sex check(sex in ('m','f')));
  sqlite> insert into emp values('1','x');
  ConstraintError: constraint failed
  </pre>

  You do get more useful information in simple circumstances:
  <pre>
  sqlite> insert into emp values('1','m');
  sqlite> insert into emp values('1','m');
  ConstraintError: column id is not unique
  sqlite> create table foo(x not null, y not null);
  sqlite> insert into foo values(null, 1);
  ConstraintError: foo.x may not be NULL
  </pre>

  <hr><i>anonymous added on 2009-10-08 11:23:51:</i><br>
  The same applies to foreign key constraints: error messages which are generated by the .genfkey triggers are also not very informative. That would be easy to fix.

  <hr><i>anonymous claiming to be BuschnicK added on 2009-11-19 12:06:32:</i><br>
  +1 for this feature request. I have quite a complex database schema maintained in parallel on various different server backends (Postgres, SQLServer, SQLite) and SQLite stands out in a bad way due to it's non-descript error messages. It would be tremendously helpful to know more information about the actual constrained that failed.


tkt_uuid: d8fe6cef65f48e06ac806f92fc54486eee841f06
type: Code_Defect
status: Closed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.6.16
resolution: Wont_Fix
title: Error return from VFS xUnlock ignored
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3946] has all the details.  The main point is that the error used to be reported but then a code change resulted in it being ignored.


tkt_uuid: 897b96d49d16dcc4dcfa5bd88ded30700e74bc94
type: Code_Defect
status: Deferred
subsystem: Extensions
priority: Immediate
severity: Minor
foundin: 3.6.18
resolution: Open
title: SQLITE_CANTOPEN when using journal_mode=persist and async IO
comment:
  Steps to reproduce:

  <ol>
  <li> not call sqlite3async_run();
  <li> open new not yet existing database;
  <li> execute PRAGMA journal_mode = PERSIST;
  <li> execute any 2 statements needing a transaction, e.g. some CREATE TABLE
  </ol>

  Tcl test code:

  <verbatim>
  sqlite3 db test.db -vfs sqlite3async
  sqlite3 db2 test.db -vfs sqlite3async
  do_test async4-3.1 {
    execsql {
      PRAGMA journal_mode = PERSIST;
      CREATE TABLE t1(a, b);
    }
  } {persist}
  do_test async4-3.2 {
    execsql { CREATE TABLE t2(a, b) } db2
  } {}
  do_test async4.3.X {
    db close
    db2 close
    sqlite3async_control halt idle
    sqlite3async_start
    sqlite3async_wait
  } {}
  </verbatim>

  Test case async4-3.2 fails with "Error: unable to open database file".

  <hr><i>dan added on 2009-10-19 11:43:11:</i><br>
  This is a real bug, and it is possible to fix it. However, the nature of the fix is such that it would quite possibly introduce other, more serious bugs (the kind that can cause database corruption). For this reason, this will not be fixed in the short term.

  For now, the resolution is to consider async IO and persistent-journal mode incompatible.


tkt_uuid: c060923a5422590b3734eb92eae0c94934895b68
type: Code_Defect
status: Closed
subsystem: VFS
priority: Immediate
severity: Severe
foundin: 3.6.2
resolution: Wont_Fix
title: VFS filename truncation issues
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3373]

  In essence the Unix VFS uses a hard-coded limit of 512 bytes for a fully qualified filename, truncates anything longer than that, and then gives can't open error on having stuffed the situation up.  In practise current Unix systems have maxiumum pathname lengths of 4096 bytes which means there are files that SQLite can't open.  Note the truncation happens even if a fully qualified pathname is passed in.  This has also affected mozilla.

  The Windows VFS has the same problem but it's default max length is the same as Window's.  However Windows does allow pathnames to be prefixed by "\\?\" which then uses an alternate path parser with a 32kb limit.

  <hr><i>rogerb added on 2009-10-19 05:17:39:</i><br>
  More details on how truncation also affects Windows is at [http://www.sqlite.org/cvstrac/tktview?tn=2996]


tkt_uuid: 2cb66577f600b7b657834fe3818fc7aba90d3803
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.10
resolution: Fixed
title: Diagnostic message is unintentionally printed in batch mode
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3591]

  The main() function checks a few arguments then calls process_sqliterc then processes the arguments again.  process_sqliterc prints "Loading resources from ..." but only if in interactive mode.  If the -batch parameter was supplied then it isn't looked at in the initial brief round of argument parsing which means the message can be printed in batch mode.  Additionally the message goes to stdout which means it is intermingled with real output.

  <hr><i>shane added on 2009-10-24 02:06:16:</i><br>
  Check-in [e3898e25cf].


tkt_uuid: beb2dd69ad0da769f8037e9880ceafcaa18574ec
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Important
foundin: 3.6.6.1
resolution: Fixed
title: do_meta_command inconsistent return values
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3511]

  The initial comment states that the return code should be 1 for error, 2 for exit and 0 otherwise.  Code inspection shows several places where errors are seen (look for fprintf to stderr) yet 0 is returned.  In one place whatever sqlite3_backup_step returned on error is returned.  In the dump command zErrMsg is never passed to anything yet its non-zero value is checked for indication of error instead of looking at the return code of sqlite_exec and friends.

  <hr><i>shane added on 2009-10-23 01:27:32:</i><br>
  Check-in [1ebac9eddd].


tkt_uuid: ee4b2b48f5080f7eaead1d5e08eb6af25d366916
type: Feature_Request
status: Open
subsystem: Parser
priority: Immediate
severity: Cosmetic
foundin: 3.6.4
resolution: Open
title: Prepared statement cache
comment:
  Various wrappers and users of SQLite have implemented their own statement caches.  Arguably such functionality is best implemented in SQLite itself.  A proposed implementation is an additional hash table on sqlite3* to which statements are added when sqlite3_finalize is called instead of actually finalizing them.  On prepare look in the hash table for a match and reuse else do regular prepare.  Add some code in finalize to ensure the cache doesn't grow too large.

  [http://www.sqlite.org/cvstrac/tktview?tn=3483]
  [http://www.mail-archive.com/sqlite-users@sqlite.org/msg38294.html|Mailing list post]

  <hr><i>rogerb added on 2009-10-11 07:36:09:</i><br>
  As a datapoint, using my benchmark derived from one in SQLite, I get a 15% performance improvement using a home written statement cache and a 3% performance decrease when the cache is being used but there are never any matches (statements have random values).


tkt_uuid: 4d5c039d120e10d15c9787c8ece48d54308d48df
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.1
resolution: Overcome_By_Events
title: Truncated database not detected well
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3471]

  A simple check that the file length is a multiple of the page size should catch this sort of thing.

  <pre>
  $ ./sqlite3 foo
  sqlite> pragma page_size=4096;
  sqlite> create table foo(x);
  $ dd if=foo of=foo2 bs=1024 count=1
  $ ./sqlite3 foo2
  sqlite> select * from sqlite_master;
  SQL error: malformed database schema (?)
  sqlite> pragma integrity_check;
  SQL error: malformed database schema (?)
  </pre>

  <hr><i>rogerb added on 2009-10-18 08:29:10:</i><br>
  See also [http://www.sqlite.org/cvstrac/tktview?tn=3153] where SQLite doesn't realize corruption on truncated last page.


tkt_uuid: ac55d88a33512995fcff90644a38549a0b9b6506
type: Documentation
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.17
resolution: Overcome_By_Events
title: H10336 is no longer applicable
comment:
  3.6.17 removed the restriction on shared cache and virtual tables although [http://www.sqlite.org/cvstrac/tktview?tn=3462] claims it was never there in the first place!


tkt_uuid: 7b61b6c6ce7a8c23bdc5c25c88b4bc60b0208e0d
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Low
severity: Minor
foundin: 3.6.18
resolution: Fixed
title: insert mode is incorrectly converting text to numbers
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3906]

  The shell is deciding that a string looks like a number and then generating an insert that way.  This is an unwarranted conversion but also loses information such as leading zeroes and probably also messing with strings that have digits and the letter E in them.

  <pre>
  sqlite> CREATE TABLE test_table(city TEXT, zip TEXT);
  sqlite> INSERT INTO test_table VALUES('Cambridge','02139');
  sqlite> select typeof(city), typeof(zip) from test_table;
  text|text
  sqlite> .mode insert
  sqlite> select * from test_table;
  INSERT INTO table VALUES('Cambridge',02139);
  </pre>

  <hr><i>shane added on 2009-10-22 19:38:33:</i><br>
  Check-in [ab99faca6c].

  <hr><i>shane added on 2009-10-22 21:27:32:</i><br>
  Also check-in [790402c150].


tkt_uuid: 5ee23731f15432543ba9991551c256574284910c
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Critical
foundin: 3.6.18
resolution: Fixed
title:
  assert() failure (or invalid error code) caused by opening statement transaction on read-only database with active read statements

comment:
  Attempting to execute a statement that requires a statement transaction on a read-only database while there are other read statements active is causing an
  assert() to fail. Or, if asserts are not enabled, an invalid error code to be
  returned.

  Introduced by [aec9dbd8d2].

  <hr><i>drh added on 2009-10-13 15:44:37:</i><br>
  Fixed by check-in [bc1101179abb4577417c971a7e4fbacde50e19a2].


tkt_uuid: f777251dc7abcf45d3f3e86efd09d7951132c30b
type: Code_Defect
status: Fixed
subsystem: B-Tree
priority: Immediate
severity: Critical
foundin: 3.6.19
resolution: Fixed
title: Assertion fault following ROLLBACK with running queries
comment:
  The following TCL script (run on a TCL interpreter that has the
  SQLite extension installed) generates an assertion fault:

  <verbatim>
     file delete -force b2.db
     sqlite3 db b2.db
     db eval {
       CREATE TEMP TABLE t1(x UNIQUE);
       INSERT INTO t1 VALUES(1);
       CREATE TABLE t2(x, y);
       INSERT INTO t2 VALUES(1, 2);
       CREATE TEMP TABLE t3(w, z);
     }
     proc force_rollback {} {
       catch {db eval {INSERT OR ROLLBACK INTO t1 VALUES(1)}}
     }
     db function force_rollback force_rollback
     db eval {
       BEGIN IMMEDIATE;
     }
     db eval {
       SELECT x, force_rollback(), EXISTS(SELECT 1 FROM t3 WHERE w=x) FROM t2
     } {
       puts x=$x
     }
  </verbatim>

  Bisecting show the problem was introduced by check-in [c76a366ed4].
  (2009-07-04, version 3.6.17)

  The failed assert() is the same one reported as Firefox bug 520445.

  <hr><i>drh added on 2009-10-15 17:16:17:</i><br>
  A slightly different script causes problems going all the way back to
  version 3.1.3 (2005-02-19):

  <verbatim>
    file delete -force b2.db
    sqlite3 db b2.db
    db eval {
      CREATE TEMP TABLE t1(x UNIQUE);
      INSERT INTO t1 VALUES(1);
      CREATE TABLE t2(x, y);
      INSERT INTO t2 VALUES(1, 2);
    }
    proc force_rollback {} {
      set rc [catch {db eval {INSERT OR ROLLBACK INTO t1 VALUES(1)}} msg]
    }
    db function force_rollback force_rollback
    db eval {
      BEGIN IMMEDIATE;
      CREATE TEMP TABLE t3(w, z);
    }
    db eval {
      SELECT x, force_rollback(), EXISTS(SELECT 1 FROM t3 WHERE w=x) FROM t2
    } {
      puts x=$x
    }
  </verbatim>

  <hr><i>drh added on 2009-10-15 18:06:16:</i><br>
  Here is another way to hit the same assert() that does not use
  the OR ROLLBACK clause.  We believe that this is the path that
  Firefox is using to reach the problem.  This route to the assert()
  was first opened in 3.6.17

  <verbatim>
    file delete -force b2.db
    sqlite3 db b2.db
    db eval {
      CREATE TEMP TABLE t1(x);
      CREATE TABLE t2(x);
      CREATE TABLE t3(x);
    
      INSERT INTO t1 VALUES(1);
      INSERT INTO t1 VALUES(2);
      INSERT INTO t2 VALUES(1);
      INSERT INTO t2 VALUES(2);
    }
    proc ins {} { db eval {INSERT INTO t3 VALUES('hello')} }
    db function ins ins
    
    db eval { SELECT ins() AS x FROM t2 UNION ALL SELECT ins() AS x FROM t1 }
  </verbatim>


  <hr><i>drh added on 2009-10-16 20:55:03:</i><br>
  Fixed by check-in [174477bca05d019e663fd2b7cd031189ab2e010a]


tkt_uuid: 1751725cd6d70274372ab2db5bb91dbf1a3bb6ae
type: Documentation
status: Fixed
subsystem: Documentation
priority: Immediate
severity: Important
foundin: 3.6.0
resolution: Fixed
title: sqlite_close doc should qualify advice
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=3251]

  The advice in sqlite3_close doc to use sqlite3_next_stmt to close open statements should be more qualified.  In my opinion it should be removed completely or suggested as a way of finding open statements if a close fails, but not used to finalize statements.

  If the program had not been keeping track of statements then it would leak them anyway.  If any extension module or other code is in use then it is quite possible that they have their own prepared statements and so the suggested code would result in a double free.  One example module that does this is RTree.

  <hr><i>drh added on 2009-10-20 14:24:28:</i><br>
  Check-in [f494ed38a8d3fabb7a064ba4a860d5288c5a571b].


tkt_uuid: b73fb0bd649311d133a9511b1c368d642784cfab
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.19
resolution: Fixed
title: Natural self-join defect
comment:
  The following SQL does not generate the correct result:

  <verbatim>
     CREATE TABLE t1(
       a INTEGER PRIMARY KEY,
       b TEXT
     );
     INSERT INTO t1 VALUES(1,'abc');
     INSERT INTO t1 VALUES(2,'def');
     SELECT * FROM t1 NATURAL JOIN t1;
  </verbatim>

  A simple work-around is to alias one or both of the tables 
  being joined.  For example

  <verbatim>
     SELECT * FROM t1 a NATURAL JOIN t1;
     SELECT * FROM t1 NATURAL JOIN t1 b;
     SELECT * FROM t1 a NATURAL JOIN t1 b;
  </verbatim>

  This problem appears to originate in version 3.2.2 on 2005-06-13.

  <hr><i>drh added on 2009-10-19 21:05:11:</i><br>
  See check-in [6fe6371175482d38ac4aeea994c7b20c18b7de01]


tkt_uuid: f6ee7c19869ea460bfba60670cba6176b76bc511
type: Feature_Request
status: Fixed
subsystem: Shell
priority: Immediate
severity: Cosmetic
foundin: 3.5.6
resolution: Fixed
title: Negative widths should right justify
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2933] also includes a patch.  Example use case:

  <verbatim>
    sqlite> .mode col
    sqlite> .header on
    sqlite> .width 0 -8 -8 -7 0
    sqlite> select * from my_table;

    type           count  purchase  percent  date
    ----------  --------  --------  -------  ----------
    option      20818215    495660     2.38  2008-01-28
    zoo keeper   5181793    119092      2.3  2008-01-28
    flubber        45391      5686    12.53  2008-01-28
  </verbatim>


tkt_uuid: 8d510499ccbac2f03efaf6dd0567d0e01a88a012
type: Code_Defect
status: Closed
subsystem: Parser
priority: Immediate
severity: Important
foundin: 3.5.5
resolution: Works_As_Designed
title: Repreparation errors are always returned as SQLITE_SCHEMA
comment:
  See [http://www.sqlite.org/cvstrac/tktview?tn=2922] including test and patch.

  The problem is that SQLITE_SCHEMA occurs then no matter what the error on reprepare SQLITE_SCHEMA is returned.  For example the database could be busy but you won't get SQLITE_BUSY returned.

  <hr><i>rogerb added on 2009-10-19 05:28:02:</i><br>
  This bug also affects me with APSW and I reverted to using prepare version 1 because of it.  My real world occurrence was when a collation was removed after the statement had been prepared.  If you do a fresh prepare then you get SQLERROR but on a reprepare you get SCHEMA.  Since I have a statement cache the actual library user has no knowledge of what statements are already prepared and which aren't nor should they care.

  <hr><i>rogerb added on 2009-10-20 00:38:59:</i><br>
  Also [http://www.sqlite.org/cvstrac/tktview?tn=2158]

  <hr><i>drh added on 2009-10-20 13:27:15:</i><br>
  sqlite3_prepare_v2() will return SQLITE_ERROR if there is a missing
  collating function.  But sqlite3_step() returns SQLITE_SCHEMA when it
  goes to reprepare a statement and the collating function has gone missing.
  We have decided that this is a feature, not a bug.  We want sqlite3_step()
  to return SQLITE_SCHEMA instead of SQLITE_ERROR in order to emphasize that
  the application has hit a compile-time rather than a run-time error.
  This is the unanimous decision of the developers.


tkt_uuid: d82e3f3721296e905d7e8c9dee718f71a826b0eb
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Critical
foundin: 3.4.2
resolution: Fixed
title: VACUUM corrupts database when MAIN and TEMP table names overlap.
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2879].  Confirm in 3.6.19 using shell.

  <hr><i>drh added on 2009-10-20 15:06:22:</i><br>
  When the TEMP and MAIN databases contain one or more tables with the
  same name, the VACUUM command might overwrite MAIN database content
  with information taken from the corresponding TEMP table.

  Note that this bug is unlikely to be hit because it is difficult
  to get a TEMP and MAIN table with the same name.  The only way to
  make this happen is to first create the TEMP table in one database
  connection, then create the MAIN table in a separate database connection.


  This bug has existed all the way back to version 3.1.3 (2005-02-19) and
  probably before that.  (3.1.3 is the oldest version readily at hand
  for testing.)

  Fixed by check-in [0bbd8d72ec66cbe11c2548a04ad478202b2b69b6]

  <hr><i>rogerb added on 2009-10-26 08:27:52:</i><br>
  I don't know if it is intentional but with the fix done if I do "select * from sqlite_sequence" it actually selects from temp.sqlite_sequence not main.sqlite_sequence.

  <verbatim>
  sqlite> create table perm(x integer primary key autoincrement);
  sqlite> create temporary table temp1(x integer primary key autoincrement);
  sqlite> create temporary table temp2(x integer primary key autoincrement);
  sqlite> insert into perm values(3);
  sqlite> insert into temp1 values(4);
  sqlite> insert into temp2 values(5);
  sqlite> select * from sqlite_sequence;
  temp1|4
  temp2|5
  sqlite> select * from main.sqlite_sequence;
  perm|3
  </verbatim>

  The net effect of this is that .dump dumps the values from temp.sqlite_sequence!
  <verbatim>
  DELETE FROM sqlite_sequence;
  INSERT INTO "sqlite_sequence" VALUES('temp1',4);
  INSERT INTO "sqlite_sequence" VALUES('temp2',5);
  </verbatim>

  A restore (eg doing an insert into sqlite_sequence) updates temp.sqlite_sequence not main.sqlite_sequence.


tkt_uuid: 39c85e8a4e8921b4c071ba6d43bd831aa85ec30f
type: Code_Defect
status: Fixed
subsystem: VFS
priority: Low
severity: Important
foundin: 3.5.3
resolution: Fixed
title: FormatMessage incorrectly used (Windows VFS)
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2825]

  It is possible to crash SQLite unintentionally because of this.  Mitigating the probability of the problem appearing is that it only appears to be called when there is an error with loadable extensions.

  It will also be called by xGetLastError but that routine is never called in practise.

  <hr><i>shane added on 2009-10-21 01:59:18:</i><br>
  Check-in [761396f8cb]


tkt_uuid: d214d7b114206bc7f3f1eb35ea2ce14c78c39bec
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.4.0
resolution: Overcome_By_Events
title: FTS3 virtual table fails when names are quoted
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2512]

  <verbatim>
  sqlite> create virtual table "a b c" using fts3 (t);
  SQLError: vtable constructor failed: a b c
  </verbatim>

  <hr><i>drh added on 2011-10-13 18:26:33:</i><br>
  The latest code does not exhibit this problem.


tkt_uuid: de3ae967a06e50f5ee1c9caad44d3b8833380f32
type: Feature_Request
status: Fixed
subsystem: Parser
priority: Immediate
severity: Minor
resolution: Fixed
title: Insert multiple rows at once
comment:
  There are numerous (duplicate) requests in cvstrac tracker for inserting multiple rows at once.  The syntax suggested is always the same and seems to be what is used by other databases.

  <verbatim>
  INSERT INTO foo VALUES
    (3, 4),
    (5, 6),
    (7, 8);
  </verbatim>

  <hr><i>rogerb added on 2009-10-20 06:14:00:</i><br>
  Ticket created so that an official answer can be given for the record about this since none was given in cvstrac.


tkt_uuid: f80d443a3bc6cbe9a6552af7ea85a8b7f9ef6bb1
type: Documentation
status: Closed
subsystem: B-Tree
priority: Immediate
severity: Cosmetic
resolution: Works_As_Designed
title: pragma cache size doesn report actual size (min is 10)
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2393]  See also recent mailing list activity where posters thought they had set a cache smaller than 10 and the pragma reports that is the case, but behind the scenes a minimum of 10 is enforced.

  <hr><i>drh added on 2009-10-20 13:49:06:</i><br>
  Clarified the documentation to explain that the default page cache
  implementation rounds up the suggested page cache size to a minimum
  value of 10.

  <hr><i>rogerb added on 2009-10-20 18:18:37:</i><br>
  <verbatim>
  sqlite> pragma cache_size=3;
  sqlite> pragma cache_size;
  3
  </verbatim>

  It should really be reporting 10.


tkt_uuid: 78e04e52eafa8be352a457c78edd7e8ded41e56d
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Cosmetic
foundin: 3.6.19
resolution: Fixed
title: Assertion failure dropping table with zero length name
comment:
  SQLite allows zero length table and column names:  [http://www.sqlite.org/cvstrac/tktview?tn=851]

  Dropping a zero length table results in an assertion failure:

  <verbatim>
  $ gcc -pthread -DSQLITE_DEBUG -o sqlite3 sqlite3.c shell.c -ldl
  $ ./sqlite3 
  SQLite version 3.6.19
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> create table ""("");
  sqlite> drop table "";
  sqlite3: sqlite3.c:66333: sqlite3UnlinkAndDeleteTable: Assertion `zTabName && zTabName[0]' failed.
  Aborted (core dumped)
  </verbatim>

  <hr><i>drh added on 2009-12-08 14:34:50:</i><br>
  Fixed in [7b1cfd65a6f7c85ef8f6e4622973457cff9d007c]


tkt_uuid: 61d35ac2104ec81bd1b7ce787b22a9491fceddff
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.3.13
resolution: Rejected
title: A way to tell if a journal should exist
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2224]

  To aid in troubleshooting there should be a documented way when provided with a database to determine that there should also be an accompanying journal.

  <hr><i>rogerb added on 2009-11-10 23:31:36:</i><br>
  In addition there should also be some way to check that a particular journal actually matches the database.  Consider the case of a journal being backed up and later restored but mismatching the database by a few seconds.  Replaying that journal will corrupt the database.

  A suggestion from Scott Hess on the mailing is that the journal could store the file change counter.


tkt_uuid: 36769f697da50d343034ef8ffbf4bb2945f8599b
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.3.12
resolution: Overcome_By_Events
title: termDataCmp in FTS3 should be __cdecl on Windows
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2222]

  Since termDataCmp is passed to qsort it always needs to use the C calling convention.  If SQLite is compiled with a different calling convention then there will be problems calling this function.


tkt_uuid: 4c9223e34e98d8b2eec4cb6a97827733bc86508f
type: Documentation
status: Closed
subsystem: Documentation
priority: Immediate
severity: Minor
resolution: Overcome_By_Events
title: Vacuum fails within transaction but also rolls it back
comment:
  If you perform a vacuum from within a transaction you get an error saying you cannot do that.  However it also rolls the transaction back which is unexpected and more importantly undocumented.

  [http://www.sqlite.org/cvstrac/tktview?tn=2381]  Confirmed still present in 3.6.19

  <hr><i>rogerb added on 2009-10-24 01:40:43:</i><br>
  It appears to be a little more bizarre than that.  The transaction appears to be rolled back and then a new one started.

  <hr><i>drh added on 2011-10-13 16:32:26:</i><br>
  Works now (3.7.8).


tkt_uuid: 3fe897352e8d8ceabbe9aa643f929a9a7ce988df
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.16
resolution: Fixed
title: Malformed UTF16 leads to a 2-byte buffer overread
comment:
  If a malformed UTF16 string that ends with the first half of a surrogate pair
  is passed into SQLite through functions such as sqlite3_bind_text16()
  then SQLite might read two bytes past the end of the string.  This is
  normally harmless, but if the string happens to end on a page boundary and
  the next page is unmapped, a segfault could result.

  <hr><i>shane added on 2009-10-24 01:48:05:</i><br>
  Check-in [19064d7cea].


tkt_uuid: f711d5a93771a49c02ba0a5ef25a7e9c7a6b78ca
type: Feature_Request
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
resolution: Fixed
title: Windows DLL should have versioninfo resource
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=823]  Doing this is normal and best practise for Windows libraries.

  If you right click on the DLL you should be able to see version information in the properties if this was done.  It is also usually required by Windows packaging systems.


tkt_uuid: 0c80ff9f5ad96776d95304185f675d65bdabb99f
type: Documentation
status: Fixed
subsystem: Documentation
priority: Immediate
severity: Cosmetic
resolution: Fixed
title: Quickstart missing include of stdlib.h
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=2061] Quickstart uses exit(2) but does not include header file stdlib.h that includes it which will lead to warning for someone following quickstart.  Fix is to include stdlib.h or use return instead of exit.


tkt_uuid: c276ee2e4eca7dcdbad3b6b2b3e5bd356b1e416e
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
resolution: Wont_Fix
title: null should not be allowed in primary key
comment:
  SQLite allows nulls in primary keys but should not.  This bug has been present for a long time and not immediately fixed because it could break existing databases.  The behaviour has been documented as might be fixed at some point.  This ticket is for that fix,

  [http://www.sqlite.org/cvstrac/tktview?tn=518]


tkt_uuid: 5896edbe463b6a1f0f148daeb871cd91b918b3fc
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.4.0
resolution: Rejected
title: List available SQL functions
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=1864]

  Note that this has to be implemented as part of the SQLite core since there are no public apis for getting function information.


tkt_uuid: bcff482dd669cda650ab17dc92a5b9f108933ed4
type: Feature_Request
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.3.6
resolution: Fixed
title: Explicit extended error code for deadlock with sqlite_busy
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=1837]

  [http://www.mail-archive.com/sqlite-users%40sqlite.org/msg15979.html]

  An extended error code alongside SQLITE_BUSY would allow distinguishing between potential deadlock detected and plain old things are just busy.


tkt_uuid: 65a1fd4bab8521bd70f91e5383aac292271891e6
type: Feature_Request
status: Closed
subsystem: Shell
priority: Immediate
severity: Cosmetic
resolution: Rejected
title: Shell should call rl_readline_name
comment:
  [http://www.sqlite.org/cvstrac/tktview?tn=1785]

  If a user has customized inputrc then they can't have shell specific sections.  Readline documentation says:

  <verbatim>
  Variable: const char * rl_readline_name
      This variable is set to a unique name by each application using Readline. The value allows conditional parsing of the inputrc file (see section 1.3.2 Conditional Init Constructs). 
  </verbatim>


tkt_uuid: 0841d0f524915dcff9e41f1f31034583c9b89906
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
resolution: Wont_Fix
title: Exclusive transactions do not work if same database multiply attached
comment:
  [http://www.sqlite.org/cvstrac/tktedit?tn=2653]

  <verbatim>
  sqlite> attach 'testdb' as db1;
  sqlite> attach 'testdb' as db2;
  sqlite> create table db1.foo(x);
  sqlite> begin exclusive  ;
  BusyError: database is locked
  </verbatim>


tkt_uuid: 293860ac261a7288deea530337838cbcbf4fb9c4
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
resolution: Rejected
title: Allow triggers to reference other databases
comment:
  There have been various requests over the years for triggers to reference tables other than the database they are defined in.

  One example [http://www.sqlite.org/cvstrac/tktview?tn=1689]

  <hr><i>rogerb added on 2009-10-27 19:51:30:</i><br>
  Also [http://www.sqlite.org/cvstrac/tktview?tn=1550]


tkt_uuid: 6b00e0a34c8abd71093ea35df3f8ce9a8012aa0d
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.6.16
resolution: Fixed
title: The great OP_If bug
comment:
  The code generator for EXISTS and SELECT expressions was storing the
  number of the register that holds the result in the Expr.iColumn field.
  However, Expr.iColumn is a 16-bit signed integer.  Hence, for very complex
  statements that use more than 32768 registers prior to running the EXISTS
  or SELECT, the register number will overflow and possibly result in using
  a negative number for an array index and causing a subsequent malfunction
  or crash.

  Solutions to this problem include:

    1.  Make the Expr.iColumn field a 32-bit signed integer.  (Version 3.6.16.1)

    2.  Do not store the result register number in Expr but instead return the
        result register number as a function return value from the code
        generator routines that evaluate EXISTS and SELECT.  (Version 3.6.20)

  <hr><i>drh added on 2009-10-30 14:05:23:</i><br>
  Fixed by check-in [65a1f1334d] on the 3.6.16 branch.

  Fixed by check-in [7253f8fad1] on the trunk.


tkt_uuid: 0bf974bdf9e7bce5e4b732147ba000a3f8f630e6
type: Code_Defect
status: Fixed
subsystem: Extensions
priority: Immediate
severity: Minor
foundin: 3.6.20
resolution: Fixed
title: backup api asserts with zero length source file
comment:
  ensure 'test.db' and 'foo' do NOT exist, or are zero in length.<br>
  with the CLI:<br>
  sqlite3 test.db<br>
  >.restore foo<br>

  introduced with [866216c0b6]

  <hr><i>drh added on 2009-11-06 04:15:47:</i><br>
  This actually worked correctly, except for some assert() statements.
  Check-in [ddb71cd9ed395804a13dc136bb7688a7627c798f] fixes those faulty
  assert() statements so that everything is working now.

  Test cases added to TH3.


tkt_uuid: 33fdaab1a048cbdaf0e72043b70361c60613d9b4
type: Feature_Request
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Minor
foundin: 3.6.20
resolution: Rejected
title: Need to cancel an integrity check in progress with sqlite3_interrupt
comment:
  [http://article.gmane.org/gmane.comp.db.sqlite.general/52203]

  sqlite3_interrupt does not appear to cancel an integrity check (pragma
  integrity_check or quick_check) in progress. Given how long these
  checks can take on big DBs, it's nice to be able to abort them (if
  your app is asked to exit for example).

  I've hacked in my own support for doing this, but would love to see an
  official version committed for a future release.

  The simple fix I put in place was to add:

  <verbatim>
  sqlite3_snprintf(sizeof(zContext), zContext, "Page %d: ", iPage);
  /* allow checking to be aborted */
  if (pCheck->pBt->db->u1.isInterrupted)
  {
   checkAppendMsg(pCheck, zContext, "Check interrupted");
   return -1;
  }
  </verbatim>
  ...near the top of: checkTreePage(...)

  In practice this causes several other error messages to get appended
  to the check results as well, and the actual statement returns
  SQLITE_ROW. A better solution would be one that causes the execution
  to return  SQLITE_INTERRUPT, but this is good enough for my case.


tkt_uuid: aeff892c57d408b7a470938c61e803cc69635aa4
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.6.20
private_contact: 26ed8afc9e6e35b4ac4c1082f7be8344fd6c369c
resolution: Fixed
title: -batch mode not creating DB in CLI
comment:
  User Ken Zalewski reported on the mailing list:
  <pre>
  > It seems that the "-batch" option in 3.6.20 has stopped working.
  > If I run "sqlite3 -batch blah.db < create_tables.sql"
  > the blah.db database is never created.
  >
  > In versions up to and including 3.6.19, the blah.db file would
  > properly be created.
  >
  > In fact, the -batch option, even when used with an already-existing
  > database, seems to have no effect.  If I run "sqlite3 -batch
  > existing.db < create_tables.sql", no tables are actually created.
  </pre>

  <hr><i>shane added on 2009-11-10 17:03:35:</i><br>
  verified as a bug

  <hr><i>shane added on 2009-11-10 17:07:21:</i><br>
  Check-in [8097c64acf].


tkt_uuid: 517e206abde2f985442019b7cf95f6a98b0a80ca
type: Code_Defect
status: Fixed
subsystem: Utilities
priority: Immediate
severity: Important
foundin: 3.6.20
private_contact: 22d2ebb9a6dbfa47bd4b558ba855d0331d912298
resolution: Fixed
title: Fix sqlite3_analyzer
comment:
  Building the sqlite3_analyzer from 3.6.20 sources results in a tool that reports incorrect results (negative numbers, etc).  Comment from DRH: "Changes in our implementation prevent sqlite3_analyzer.exe from working in the near-term.  We are planning to look into fixing it long-range." This ticket is intended to track fixing it.


tkt_uuid: 564d412f15a000f3cbc9b637a850d98bfced5e6a
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
resolution: Fixed
title: OOM on autovacuum of in-memory database causes problems.
comment:
  If an out-of-memory error occurs during a COMMIT of a in-memory autovacuumed
  database, it might leave the database in a state where it cannot be rolled back.

  <hr><i>drh added on 2009-11-20 18:51:24:</i><br>
  See the following check-ins:

  [9a429349ccc2fa9acd28365a86578f602e87dafb]
  [e493b093f8ca722c3160b32a16fb615023978dc9]
  [2f42f91fe65b0b21671936013df08037091f0cc6]


tkt_uuid: 16fbf14cb24742d9ed6624bc1148c0e25daefdd7
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
private_contact: fe61b5fb4426aec8e2a3f5955ebecd31d393d979
resolution: Fixed
title: Faulty VDBE code generated for certain third-party virtual tables
comment:
  Consider a virtual table implementation where the xBestIndex() method
  indicates that one or more columns are needed by xFilter() but that the
  constraints on those columns should still be tested externally to the
  virtual table.
  If such a virtual table is used as part of a join, then it is possible
  that the external constraint checks might make use of stale register
  contents that were computed for the VFilter opcode, leading to an
  incorrect answer.

  Neither FTS3 nor RTree work as described above and so are not susceptible to 
  this bug.

  <hr><i>drh added on 2009-11-23 21:24:42:</i><br>
  Fixed by check-in [83dc7d38c2695d1b2a7d4866a4df9be9e41d1abe]


tkt_uuid: 3671265894a6321d5269a461259c60e38d868fad
type: Feature_Request
status: Closed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.6.20
resolution: Rejected
title: No error reporting on file open failure
comment:
  If the shell fails to open a file it merely reports that file opening failed without saying why.  In the various places where fopen is called and fails there should be a subsequent call to perror() to say why the open failed.


tkt_uuid: 92af7da36b6fbdaedca884776f2db3307308764d
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.6.20
resolution: Fixed
title: shell won't import large files
comment:
  See [http://thread.gmane.org/gmane.comp.db.sqlite.general/52859]

  The shell is not compiled with large file support by default.  Consequently it fails to open files bigger than 2GB (in a 32 bit environment).


tkt_uuid: 13137dccf36902cdd4e20c11d857c7ad7f5d8777
type: Feature_Request
status: Closed
subsystem: Extensions
priority: Immediate
severity: Minor
foundin: 3.6.21
private_contact: 656f4be4cb39ac37cafcfa0f0dc1750abfe89a4d
resolution: Rejected
title: FTS3 and last_insert_rowid()
comment:
  The following session illustrates the problem.

  <verbatim>
     SQLite version 3.6.21
     Enter ".help" for instructions
     Enter SQL statements terminated with a ";"
     sqlite> CREATE VIRTUAL TABLE t1 USING fts3;
     sqlite> INSERT INTO t1(rowid, content) VALUES(123, 'some text');
     sqlite> SELECT last_insert_rowid();
     1
     sqlite> SELECT rowid FROM t1;
     123
  </verbatim>

  FTS3 writes data to the database from with the virtual table xSync() callback. This is setting the value returned by last_insert_rowid().

  <hr><i>dan added on 2010-03-25 04:43:22:</i><br>
  Still in 3.6.23.


tkt_uuid: abe728bbc311d81334dae9762f0db87c07a98f79
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Critical
foundin: 3.6.0
private_contact: 86f63f1d588d613da8dd9bb600900fecf7d1b926
resolution: Fixed
title: Crash committing multi-file transaction including temp file
comment:
  Assuming the main database is not in-memory, the following 
  script causes a crash:

  <verbatim>
    ATTACH '' AS aux2;
    BEGIN;
      CREATE TABLE main.t1(x);
      CREATE TABLE aux2.t2(x);
    COMMIT;
  </verbatim>

  The problem is that in vdbeaux.c during multi-file commit, a NULL value
  returned by sqlite3BtreeJournalName() for the temporary database is being
  dereferenced.

  <hr><i>dan added on 2009-12-08 10:45:29:</i><br>

  Introduced by [5173b3e816] (version 3.6.0).


  <hr><i>drh added on 2009-12-08 13:46:27:</i><br>
  Fixed in [f43e37e0a4cc7c15c996f4404dc775636396567d]


tkt_uuid: f74beaabde9d153ca905e0377dc50feb90bdd46b
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.21
resolution: Fixed
title: Problem with 3-way joins and the USING clause
comment:
  In SQLite, the first SELECT query below is equivalent to the second. But it should be equivalent to the third.

  <verbatim>
  SQLite version 3.6.21
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a int);
  sqlite> CREATE TABLE t2(a int);
  sqlite> CREATE TABLE t3(a int, b int);
  sqlite> INSERT INTO t1 VALUES('abc');
  sqlite> INSERT INTO t3 VALUES('abc', 'def');
  sqlite> SELECT * FROM t1 LEFT JOIN t2 LEFT JOIN t3 USING(a);
  abc||
  sqlite> SELECT * FROM t1 LEFT JOIN t2 LEFT JOIN t3 ON t3.a=t2.a;
  abc|||
  sqlite> SELECT * FROM t1 LEFT JOIN t2 LEFT JOIN t3 ON t3.a=t1.a;
  abc||abc|def
  </verbatim>

  Also, the following should not fail:

  <verbatim>
  SQLite version 3.6.21
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(w,x);
  sqlite> CREATE TABLE t2(x,y);
  sqlite> CREATE TABLE t3(w,z);
  sqlite> SELECT * FROM t1 JOIN t2 USING(x) JOIN t3 USING(w);
  Error: cannot join using column w - column not present in both tables
  </verbatim>

  <hr><i>dan added on 2009-12-09 16:27:56:</i><br>
  These problems have been present since at least 3.5.0.


  <hr><i>drh added on 2009-12-09 16:42:59:</i><br>
  Here is an example of the same problem using NATURAL JOIN.

  <verbatim>
    CREATE TABLE t1(a,x,y);
    INSERT INTO t1 VALUES(1,91,92);
    INSERT INTO t1 VALUES(2,93,94);
    CREATE TABLE t2(b,y,z);
    INSERT INTO t2 VALUES(3,92,93);
    INSERT INTO t2 VALUES(4,94,95);
    CREATE TABLE t3(c,x,z);
    INSERT INTO t3 VALUES(5,91,93);
    INSERT INTO t3 VALUES(6,99,95);
    
    SELECT * FROM t1 NATURAL JOIN t2 NATURAL JOIN t3;
  </verbatim>

  The result set in the final query above includes two "x" columns.  In
  all cases above, the problem is that USING and NATURAL JOIN are only
  looking for columns in the one table to the immediate left of the
  join operator, but they should (apparently) be looking at all tables
  to the left of the join operator and giving precedence to the left-most
  table.

  <hr><i>drh added on 2009-12-09 18:23:30:</i><br>
  Fixed by [b558e96f0a] 


tkt_uuid: 31338dca7e3ddc9e7397f881218a23a9d4b251cd
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.21
resolution: Fixed
title: OR operator in WHERE clause gives wrong answer when indexed
comment:
  The following code shows two identical queries, one before indices were
  created and the other after.  The presence of indices should never change
  the result of a query, but in this case it does:

  <blockquote><verbatim>
  CREATE TABLE t1(x);
  CREATE TABLE t2(y);
  INSERT INTO t1 VALUES(111);
  INSERT INTO t1 VALUES(222);
  INSERT INTO t2 VALUES(333);
  INSERT INTO t2 VALUES(444);

  SELECT 'noindex', * FROM t1, t2
   WHERE (x=111 AND y!=444) OR x=222;

  CREATE INDEX t1x ON t1(x);

  SELECT 'w/index', * FROM t1, t2
   WHERE (x=111 AND y!=444) OR x=222;
  </verbatim></blockquote>

  This is problem is observed in SQLite versions 3.6.18 through 3.6.21.
  Versions 3.6.17 and earlier appear to work.

  <hr><i>drh added on 2009-12-16 15:18:45:</i><br>
  Problem introduced on 2009-08-13 by check-in [19f799b32f9d]

  <hr><i>drh added on 2009-12-16 16:40:47:</i><br>
  Another interesting test case.  This one gives an assertion fault.

  <blockquote><verbatim>
  CREATE TABLE t1(v,w);
  CREATE TABLE t2(x,y);
  CREATE TABLE t3(z);
  INSERT INTO t1 VALUES(111,222);
  INSERT INTO t1 VALUES(333,444);
  INSERT INTO t2 VALUES(222,333);
  INSERT INTO t2 VALUES(444,555);
  INSERT INTO t3 VALUES(888);
  INSERT INTO t3 VALUES(999);

  CREATE INDEX t1v ON t1(v);
  CREATE INDEX t2x ON t2(x);

  SELECT * FROM t1, t2, t3
   WHERE (v=111 AND x=w AND z!=999) OR (v=333 AND x=444);
  </verbatim></blockquote>

  <hr><i>drh added on 2009-12-16 22:11:58:</i><br>
  Fixed by check-in [2c2de252666662f5459904fc33a9f2956cbff23c]


tkt_uuid: 356c885b0b2c50d6ab6ed9642b41a08ab8f6c745
type: Code_Defect
status: Fixed
subsystem: Other
priority: Immediate
severity: Minor
foundin: 3.6.21
private_contact: 2b7f554184c58a40ee0000146b1d214ecfd70237
resolution: Fixed
title: sqlite3_complete() should return 0 on empty string
comment:
  sqlite3_complete() should return 0 on empty string as the string does not contain "one or more non-whitespace tokens."

     [http://sqlite.org/hlr10000.html#H10511]

  <hr><i>shane added on 2009-12-17 22:18:05:</i><br>
  Fixed by check-in [76eca7295c].


tkt_uuid: eb5548a8496477212d80c4aa4bcb035d5e330211
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.6.21
resolution: Fixed
title: CAST in WHERE clause causes inconsistent query results
comment:
  In the following code, the two queries get different answers:

  <blockquote><verbatim>
  CREATE TABLE t1(x);
  INSERT INTO t1 VALUES('1');
  SELECT 1, x FROM t1 WHERE cast(x AS int)=1 AND x='1';
  SELECT 2, x FROM t1 WHERE x='1' AND cast(x AS int)=1;
  </verbatim></blockquote>

  The faulty behavior first appeared in SQLite version 3.6.14.

  <hr><i>drh added on 2009-12-24 16:01:52:</i><br>
  Fixed by check-in [ea4e57e1c1b6663ceb12872e31a8503f6c14e3d8].


tkt_uuid: 34d2ae1c6d08b5271ba5e5592936d4a1d913ffe3
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
resolution: Fixed
title: Overloaded column named "ROWID" mishandled in triggers.
comment:
  The following SQL shows that the normal column name "rowid" is interpreted
  as the ROWID inside a trigger:

  <blockquote><verbatim>
  CREATE TABLE t1(rowid, nid);
  CREATE TABLE LOG(a,b);
  CREATE TRIGGER r1 AFTER INSERT ON t1 BEGIN
    INSERT INTO log VALUES(new.rowid, new.nid);
  END;
  INSERT INTO t1 VALUES(100,200);
  SELECT * FROM log;
  </verbatim></blockquote>

  This problem appears to be associated with the introduction of recursive
  triggers in version 3.6.18.

  <hr><i>drh added on 2009-12-29 23:39:46:</i><br>
  Fixed by check-in [1a0e5fa9f0ff792c678c99d49960601a0cef925b]


tkt_uuid: 0d676cd6e67b2a8d7e97b46a89537c51325524ee
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.18
private_contact: 4cbcf594e19626e5f476ac9fcae4c6929add5251
resolution: Duplicate
title: Triggers: new/old.oid is interpreted as the IPK, even if there is another column "oid"
comment:
  Within a trigger body, a reference to new.oid is always interpreted as a reference to the rowid. However, if there is an explicitly declared column named "oid", then "new.oid" should refer to the explicit column. Example:

  <verbatim>
  SQLite version 3.6.21
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE foo(k INTEGER PRIMARY KEY, oid INTEGER);
  sqlite> CREATE TABLE log(a, b);
  sqlite> 
  sqlite> CREATE TRIGGER tr1 AFTER INSERT ON foo BEGIN
     ...>   INSERT INTO log VALUES(new.k, new.oid);
     ...> END;
  sqlite> INSERT INTO foo VALUES(100, 200);
  sqlite> SELECT a, b FROM log;
  100|100
  </verbatim>

  The result of the above SELECT should be "100|200".

  <hr><i>dan added on 2009-12-30 07:41:28:</i><br>
  Dup of [34d2ae1c6d].


tkt_uuid: 360c6073e1972c0beecaeb74ca64efb3961c2739
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.22
resolution: Fixed
title: Aggregate MAX() function with COLLATE clause
comment:
  Consider the following SQL:

  <blockquote><verbatim>
  CREATE TABLE t1(x);
  INSERT INTO t1 VALUES('abc');
  INSERT INTO t1 VALUES('ABC');
  INSERT INTO t1 VALUES('BCD');
  SELECT max(x) FROM t1;
  SELECT max(x COLLATE nocase) FROM t1;
  </verbatim></blockquote>

  The two queries at the end return the expected result, <b>abc</b> and 
  <b>BCD</b> respectively.  But now look at these queries:

  <blockquote><verbatim>
  SELECT max(x), max(x COLLATE nocase) FROM t1;
  SELECT max(x COLLATE nocase), max(x) FROM t1;
  </verbatim></blockquote>

  We expect the first query to return <b>abc BCD</b> and the second
  to return <b>BCD abc</b>.  But instead, we get <b>abc abc</b> and
  <b>BCD BCD</b>.

  Here is a guess at what is going wrong:  The aggregate analysis thinks
  that "max(x)" and "max(x COLLATE nocase)" are the same function and
  is coalescing them into a single calculation.

  This problem has been in every version of SQLite going back to
  version 3.3.13 (2007-02-13) which was the first version to support
  the COLLATE operator.  In other words, it has never worked.

  This issue was discovered while writing test cases to verify that the
  actual behavior of the aggregate MAX() function matches the documented
  behavior.

  <hr><i>drh added on 2010-01-07 13:25:58:</i><br>
  Here is another manifestation of the same problem:

  <blockquote><verbatim>
  CREATE TABLE t1(x CHECK( x != 'abc' ));
  INSERT INTO t1 VALUES('ABC');
  CREATE TABLE t2(x CHECK( x != 'abc' COLLATE nocase));
  INSERT INTO t2 SELECT * FROM t1;
  </verbatim></blockquote>

  The insert into T2 is allowed even though the CHECK constraint is violated.
  But if the final INSERT is like this:

  <blockquote><verbatim>
  INSERT INTO t2 SELECT x FROM t1;
  </verbatim></blockquote>

  Then the CHECK constraint fires and the insert is disallowed.

  <hr><i>drh added on 2010-01-07 15:17:33:</i><br>
  Check-in [44bb1bfe5dedd8054ddd933941ee4112ed8d3b68]


tkt_uuid: eb942c64a157c085b8eb2e26507193ce5d6c23a1
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.16
resolution: Fixed
title: PRAGMA reverse_unordered_selects causes schema parse to fail
comment:
  Suppose a database is created as follows:

  <verbatim>
    CREATE TABLE t1(x);
    CREATE INDEX i1 ON t1(x);
  </verbatim>

  Then you close the database and reopen it, and then run the following SQL:

  <verbatim>
    PRAGMA reverse_unordered_selects = ON;
    SELECT * FROM t1;
  </verbatim>

  This results in the following error:

  <verbatim>
    malformed database schema (i1) - no such table: main.t1
  </verbatim>

  We postulate that the SELECT statements used to parse the schema in 
  the sqlite_master table needs to have an "ORDER BY rowid" clause added.

  The work-around is to omit the PRAGMA until after the schema is parsed.

  This bug seems to be present in all versions of SQLite since
  3.6.12 (2009-03-31), which was the first version to include the
  reverse_unordered_selects pragma.

  <hr><i>drh added on 2010-01-13 13:24:21:</i><br>
  This same issue comes up when trying VACUUM with the reverse_unordered_selects
  pragma enabled.  Both issues were fixed with check-in 
  [0209f17180065d9c213b6c437be1c3c40660665f].


tkt_uuid: 1258875e07553c989030934c1125afdf5c79e99b
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
resolution: Fixed
title: Undefined collating sequence on LIKE causes segfault
comment:
  Suppose a table is created with a application-defined collating sequence:

  <verbatim>
      CREATE TABLE t1(x TEXT COLLATE appdef);
  </verbatim>

  Then the same database is opened in a separate database connection where
  the "appdef" collating sequence is undefined.  Then the column is used as
  the left-hand side of a LIKE operator:

  <verbatim>
      SELECT * FROM t1 WHERE x LIKE 'abc%';
  </verbatim>

  This causes a segfault.

  The problem appears to have been introduced by check-in 
  [9854ad00aed08793ae7ba3c2cfbab7a2dba4dcb8] on 2009-10-22
  which was part of release 3.6.20.

  <hr><i>drh added on 2010-01-21 23:13:44:</i><br>
  Fixed by check-in [a82e6b4585b9ddba581204976cd5ea8704339185].
  The test case is in TH3.


tkt_uuid: 8905af11a0ade7ad81cf339ee6fc22eb31565977
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.22
resolution: Works_As_Designed
title: NOT NULL interferes with ON CONFLICT REPLACE
comment:
  In the following code, the ON CONFLICT REPLACE declaration on column x is
  ignored:

  <pre>
     CREATE TABLE t1(x PRIMARY KEY NOT NULL ON CONFLICT REPLACE, y);
     INSERT INTO t1 VALUES(1,2);
     INSERT INTO t1 VALUES(1,3);  -- ought to work, but throws an error
  </pre>

  Curiously, reversing the order of the NOT NULL and ON CONFLICT clauses fixes
  the problem:

  <pre>
     CREATE TABLE t2(x PRIMARY KEY ON CONFLICT REPLACE NOT NULL, y);
     INSERT INTO t2 VALUES(1,2);
     INSERT INTO t2 VALUES(1,3);
     SELECT * FROM t2;  -- output: 1|3
  </pre>

  <hr><i>drh added on 2010-02-02 20:43:53:</i><br>
  This problem is present in all versions of SQLite back through
  version 3.1.3 (2005-02-19), which is as far back as we tested for it.
  Our guess is that it has never worked correctly.

  <hr><i>drh added on 2010-02-02 20:53:40:</i><br>
  This is not a bug after all.  The behavior is as defined and as expected.

  With PRIMARY KEY NOT NULL ON CONFLICT REPLACE, the ON CONFLICT clauses
  modifies the NOT NULL constraint, not the PRIMARY KEY constraint.  The
  ON CONFLICT clause is applied to whatever constraint comes immediately
  before it.  Hence, to get ON CONFLICT to modify PRIMARY KEY, you must
  put the ON CONFICT clause immediately after PRIMARY KEY.


tkt_uuid: 985771e1161200ae5eac3162686ea6711c035d08
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
resolution: Fixed
title: CREATE TRIGGER confused by TEMP/MAIN table name overlap
comment:
  When both a main database table and a TEMP table have the same name, and
  a main database trigge is created on the main table, the trigger is 
  incorrectly bound to the TEMP table.  For example:

  <verbatim>
     CREATE TABLE t1(x);
     CREATE TEMP TABLE t1(x);
     CREATE TABLE t2(z);
     CREATE TRIGGER main.r1 AFTER INSERT ON t1 BEGIN
       INSERT INTO t2 VALUES(10000 + new.x);
     END;
     INSERT INTO main.t1 VALUES(3);
     INSERT INTO temp.t1 VALUES(4);
     SELECT * FROM t2;
   </verbatim>

  The r1 trigger fires when the value 4 is inserted into the temp.t1 table, 
  rather than when value 3 is inserted into main.t1.


tkt_uuid: 13336e9c3c8c3fe955832e128529a93d6029ad76
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
resolution: Fixed
title: ALTER TABLE RENAME fails to update foreign key references
comment:
  The ALTER TABLE RENAME command does not update foreign key references on
  ATTACH-ed or TEMP tables.  For example:

  <verbatim>
     PRAGMA foreign_keys=ON;
     ATTACH ':memory:' AS mem;
     CREATE TABLE mem.t1(x PRIMARY KEY);
     CREATE TABLE mem.t2(y REFERENCES t1);
     INSERT INTO t1 VALUES(1);
     INSERT INTO t2 VALUES(1);
     ALTER TABLE t1 RENAME TO t3;
     INSERT INTO t2 VALUES(1);
  </verbatim>

  The final INSERT should work, but it instead complains that table mem.t1 does
  not exist.


tkt_uuid: e9a9fde1f4a2c1ef4e5b06ede806728df84c3917
type: Code_Defect
status: Fixed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.6.22
private_contact: f4c53e96c2694db37fe13311b293f066f80f8e4a
resolution: Fixed
title: rtree module malfunctions if db page-size is changed
comment:
  The following script causes a buffer overread:

  <verbatim>
    PRAGMA page_size = 1024;
    CREATE VIRTUAL TABLE rt USING rtree(id, x1, x2, y1, y2);
    INSERT INTO rt VALUES(1, 2, 3, 4, 5);
    PRAGMA page_size = 2048;
    VACUUM;
    SELECT * FROM rt;
  </verbatim>
  as the r-tree code assumes that the node-size used in the r-tree structures is a function of the database page size.


tkt_uuid: da1151f97df244a1f32df704b79d4cadf17e858a
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.22
resolution: Fixed
title: Assertion fault on VACUUM in an auto_vacuumed database.
comment:
  Running the following SQL against an empty database generates an
  assertion fault on SQLite version 3.6.17 through 3.6.22.

  <blockquote><pre>
  PRAGMA auto_vacuum=FULL;
  CREATE TABLE t1(
    c000, c001, c002, c003, c004, c005, c006, c007, c008, c009,
    c010, c011, c012, c013, c014, c015, c016, c017, c018, c019,
    c020, c021, c022, c023, c024, c025, c026, c027, c028, c029,
    c030, c031, c032, c033, c034, c035, c036, c037, c038, c039,
    c040, c041, c042, c043, c044, c045, c046, c047, c048, c049,
    c050, c051, c052, c053, c054, c055, c056, c057, c058, c059,
    c060, c061, c062, c063, c064, c065, c066, c067, c068, c069,
    c070, c071, c072, c073, c074, c075, c076, c077, c078, c079,
    c080, c081, c082, c083, c084, c085, c086, c087, c088, c089,
    c090, c091, c092, c093, c094, c095, c096, c097, c098, c099,
    c100, c101, c102, c103, c104, c105, c106, c107, c108, c109,
    c110, c111, c112, c113, c114, c115, c116, c117, c118, c119,
    c120, c121, c122, c123, c124, c125, c126, c127, c128, c129,
    c130, c131, c132, c133, c134, c135, c136, c137, c138, c139,
    c140, c141, c142, c143, c144, c145, c146, c147, c148, c149
  );
  CREATE TABLE t2(
    c000, c001, c002, c003, c004, c005, c006, c007, c008, c009,
    c010, c011, c012, c013, c014, c015, c016, c017, c018, c019,
    c020, c021, c022, c023, c024, c025, c026, c027, c028, c029,
    c030, c031, c032, c033, c034, c035, c036, c037, c038, c039,
    c040, c041, c042, c043, c044, c045, c046, c047, c048, c049,
    c050, c051, c052, c053, c054, c055, c056, c057, c058, c059,
    c060, c061, c062, c063, c064, c065, c066, c067, c068, c069,
    c070, c071, c072, c073, c074, c075, c076, c077, c078, c079,
    c080, c081, c082, c083, c084, c085, c086, c087, c088, c089,
    c090, c091, c092, c093, c094, c095, c096, c097, c098, c099,
    c100, c101, c102, c103, c104, c105, c106, c107, c108, c109,
    c110, c111, c112, c113, c114, c115, c116, c117, c118, c119,
    c120, c121, c122, c123, c124, c125, c126, c127, c128, c129,
    c130, c131, c132, c133, c134, c135, c136, c137, c138, c139,
    c140, c141, c142, c143, c144, c145, c146, c147, c148, c149
  );
  VACUUM;
  </pre></blockquote>


tkt_uuid: d2f02d37f52bfe23e421f2c60fbb8586ac76ff01
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.22
resolution: Fixed
title: Assertion fault on UPDATE involving two virtual tables.
comment:
  In a shell that supports both FTS3 and RTREE, the UPDATE in the following
  code causes an assertion fault:

  <blockquote><pre>
    CREATE VIRTUAL TABLE t1 USING rtree(id, x0, y0, x1, y1);
    CREATE VIRTUAL TABLE t2 USING fts3(title, content);
    UPDATE t2 SET docid=(SELECT id FROM t1 WHERE x0>:lower AND x1<:upper)
     WHERE t2 MATCH 'this is a test';
  </pre></blockquote>

  This problem was originally reported by Gunter Hick on the sqlite-users
  mailing list.

  <hr><i>drh added on 2010-02-24 14:03:03:</i><br>
  The problem was introduced by check-in [f678ed34faab88f4e126e553119773386c7c84fa]
  on 2009-05-19.


tkt_uuid: ebdbadade5b33ff34b90fe9f1f5761fa6fc379e1
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
resolution: Fixed
title: LEFT JOIN incorrect when ON clause does not reference right table.
comment:
  As reported by Mark Brand on the sqlite-users mailing list, the following
  SQL yields an incorrect answer:

  <blockquote><verbatim>
  CREATE TABLE currency (
   cur CHAR(3),
   PRIMARY KEY (cur)
  );

  CREATE TABLE exchange (
   cur1 CHAR(3),
   cur2 CHAR(3),
   rate REAL,
   PRIMARY KEY (cur1, cur2)
  );

  INSERT INTO currency (cur) VALUES ('EUR');
  INSERT INTO currency (cur) VALUES ('GBP');
  INSERT INTO currency (cur) VALUES ('USD');

  INSERT INTO exchange (cur1, cur2, rate) VALUES ('EUR', 'GBP', 0.85);
  INSERT INTO exchange (cur1, cur2, rate) VALUES ('GBP', 'EUR', 1/0.85);

  SELECT c1.cur cur1, c2.cur cur2, COALESCE(self.rate, x.rate) rate
  FROM currency c1
  CROSS JOIN currency c2
  LEFT JOIN exchange x
     ON x.cur1=c1.cur
     AND x.cur2=c2.cur
  LEFT JOIN (SELECT 1 rate) self
     ON c1.cur=c2.cur;
  </verbatim></blockquote>

  The output consists of only three rows -- only rows for which the
  right table of the second LEFT JOIN is non-null.  Nine rows should
  be returned.

  This is a long-standing bug in SQLite.  The same incorrect answer
  is returned by all versions from 3.6.22 going back to 3.1.3
  ([2005-02-19]).

  The work-around is simple.  Move the ON clause up to the CROSS JOIN
  where it belongs.  But the problem should still be fixed.


tkt_uuid: 9abd6aa831b0df3b8d326387348b3f81b9bc3e8f
type: Code_Defect
status: Closed
subsystem: Utilities
priority: Immediate
severity: Minor
foundin: 3.6.22
private_contact: 6afbbbcb76d6fcdc913e3d93238f94e85534f2c0
resolution: Fixed
title: buffer overread in sqlite3_printf functions.
comment:
  The call to sqlite3_mprintf() in this program:

  <verbatim>
  int main(int a, char **b){
    char *zBuf = malloc(3);
    zBuf[0] = 'a';
    zBuf[1] = 'b';
    zBuf[2] = 'c';
    sqlite3_free( sqlite3_mprintf("%.3q", zBuf) );
    free(zBuf);
    return 0;
  }
  </verbatim>

  dereferences (zBuf+3), which it probably should not do. The linux man page
  for printf() says of the %s conversion type "... if a precision is specified, no more than the number specified are written. If a precision  is given, no null byte need be present ...". sqlite3_printf() functions should probably follow this example.

  This defect may have caused a warning in a debug build of firefox.


tkt_uuid: aa92c76cd48c1a7854ab9a6d5e48e9d967334d76
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Minor
foundin: 3.6.22
private_contact: b91775dfc0e51721dd6529ca1e89e1a462f28cbb
resolution: Fixed
title: Problem with GROUP BY and the distinction between real and integer values
comment:
  If the value of a column is used in more than one expression in the select-list of an aggregate query, then any affinity applied as part of one expression may affect the value used as an input to another. The following examples illustrate this:

  <verbatim>
  do_test x-1.1 {
    execsql {
      CREATE TABLE t1(a REAL);
      INSERT INTO t1 VALUES(44.0);
      INSERT INTO t1 VALUES(56.0);
    }
  } {}

  do_test x-1.2 {
    execsql {
      pragma vdbe_trace = 0;
      SELECT (CASE WHEN a=0 THEN 0 ELSE (a + 25) / 50 END) AS categ, count(*)
      FROM t1 GROUP BY categ
    }
  } {1.38 1 1.62 1}

  do_test x-2.1 {
    execsql {
      CREATE TABLE t2(a REAL);
      INSERT INTO t2 VALUES( 2.0 );
      INSERT INTO t2 VALUES( 3.0 );
    }
  } {}

  do_test x-2.2 {
    execsql {
      SELECT (CASE WHEN a=0 THEN 'zero' ELSE a/2 END) AS t FROM t2 GROUP BY t;
    }
  } {1.0 1.5}

  do_test x-2.3 {
    execsql { SELECT a=0, typeof(a) FROM t2 }
  } {0 real 0 real}

  do_test x-2.4 {
    execsql { SELECT a=0, typeof(a) FROM t2 GROUP BY a }
  } {0 real 0 real}
  </verbatim>

  Running the script above against 3.6.22, instead of the values shown test x-1.2 returns "1 1 1 1", x-2.2 returns "1 1" and x-2.4 returns "0 integer 0 integer".


tkt_uuid: 42d45a693e6a8d1321ba7b12dc1e1992e64bfa97
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.23
resolution: Fixed
title: Spurious log message from FTS3
comment:
  FTS3 is generating a spurious log message when a connection is established
  to an existing table.  To see this, first create a virtual table:

  <blockquote><pre>
  CREATE VIRTUAL TABLE t1 USING fts3(x);
  INSERT INTO t1 VALUES('hello world');
  </pre></blockquote>

  Then in a different shell, connect to the table and display its contents.

  <blockquote><pre>
  .log stdout
  SELECT * FROM t1;
  </pre></blockquote>

  The ".log stdout" command to the shell causes log messages to be displayed
  on standard output.  The spurious message is:  "no such table: main.t1_docsize"


tkt_uuid: 170872f1c3274d86bb240fcfc101451b24b212b2
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.23
resolution: Fixed
title: Unmatched double-quote causes problems for FTS3
comment:
  An unmatched double-quote the right-hand argument to MATCH causes a strange
  error in FTS3.  Example:

  <blockquote><pre>
  CREATE VIRTUAL TABLE t1 USING fts3(x);
  INSERT INTO t1 VALUES('hello world');
  SELECT * FROM t1 WHERE t1 MATCH 'wo"*';
  </pre></blockquote>

  The SELECT statement above returns: "SQL logic error or missing database"


tkt_uuid: dd3ba6063a87a3a1f91646bb5ffee390593eeb4d
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Minor
foundin: 3.6.23
private_contact: 13df53970eb9d3001a565911b33d5b7fa511988e
resolution: Fixed
title: rtree module does not support last_insert_rowid()
comment:
  <verbatim>
  SQLite version 3.6.23
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE VIRTUAL TABLE t1 USING rtree(idx, x1, x2);
  sqlite> INSERT INTO t1 VALUES(1, 2, 3);
  sqlite> SELECT last_insert_rowid();
  582827886700920832
  </verbatim>

  The problem is that the xUpdate() method of the virtual table implementation is not setting an output variable as required.


tkt_uuid: 015d3820f24d1cbb303256be5f8a034eaaa03c7d
type: Code_Defect
status: Closed
subsystem: Pager
priority: Immediate
severity: Critical
foundin: 3.6.23
resolution: Fixed
title: Missing call to xSync() following rollback.
comment:
  When a hot-journal file or large transaction is rolled back, SQLite sometimes needs to write to the database file. It then invalidates (deletes, truncates or zeroes) the journal file.

  If an OS or power failure occurs during this or shortly afterwards, the parts of the database file that were written may be left in a corrupted state. But the journal file may have been successfully invalidated. The result is a corrupted database. The fix is to sync the database file to disk after it is modified but before the journal file is invalidated.


tkt_uuid: 9861b74ab9bf250b8850c577ebad87d6e0a1f645
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.23
resolution: Overcome_By_Events
title: Fts3 offsets() function may report SQLITE_CORRUPT with empty string.
comment:
  The last statement in this script returns SQLITE_CORRUPT.

  <verbatim>
    CREATE VIRTUAL TABLE ft USING fts3(one, two);
    INSERT INTO ft VALUES('', 'foo');
    INSERT INTO ft VALUES('foo', 'foo');
    SELECT offsets(ft) FROM ft WHERE ft MATCH 'foo';
  </verbatim>

  <hr><i>drh added on 2011-10-13 18:25:15:</i><br>
  There have been many fixes and enhancements to FTS since 3.6.23.  This
  got repaired somewhere along the way.


tkt_uuid: cbd054fa6b2254973fb9210534d556876ff1afc9
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.21
private_contact: 3af94f8b353d61214bf94318a7a64dd624245ae5
resolution: Fixed
title: "out of memory" in ANALYZE with SQLITE_ENABLE_STAT2
comment:
  With sqlite 3.6.23 with SQLITE_ENABLE_STAT2, I get
  an "out of memory" error running the following simple SQL code:

  <pre>
  CREATE TABLE dist_T (
          d_row INTEGER PRIMARY KEY,
          dist TEXT UNIQUE NOT NULL
          );
  INSERT INTO dist_T VALUES (NULL, '');
  INSERT INTO dist_T VALUES (NULL, 'a');
  INSERT INTO dist_T VALUES (NULL, 'b');
  INSERT INTO dist_T VALUES (NULL, 'c');
  INSERT INTO dist_T VALUES (NULL, 'd');
  INSERT INTO dist_T VALUES (NULL, 'e');
  INSERT INTO dist_T VALUES (NULL, 'f');
  INSERT INTO dist_T VALUES (NULL, 'g');
  INSERT INTO dist_T VALUES (NULL, 'h');
  INSERT INTO dist_T VALUES (NULL, 'i');
  ANALYZE;
  </pre>

  <hr><i>shane added on 2010-03-26 01:54:22:</i><br>
  Fixed by check-in [c33b38b59f].


tkt_uuid: 775b39dd3c2dc03f32e8204b2cc279296e9b505d
type: Code_Defect
status: Fixed
subsystem: Shell
priority: High
severity: Severe
foundin: 3.6.23
private_contact: fa2b3c70f54cbf8bb73ef135c4e497f243d1c96f
resolution: Fixed
title: Poor query planning with virtual tables, joins and ORDER BY.
comment:
  Given the following schema:

  <verbatim>
      CREATE TABLE t1(number INTEGER PRIMARY KEY, date);
      CREATE INDEX i1 ON t1(date);
      CREATE VIRTUAL TABLE ft USING fts3(title);
      CREATE TABLE bt(title);
  </verbatim>

  and the following query:

  <verbatim>
    SELECT number FROM t1, bt WHERE t1.number=bt.rowid ORDER BY date
  </verbatim>

  SQLite scans t1 using the index on column "date" and retrieves each required row from table bt using rowid-lookup. This is a good plan. But, given:

  <verbatim>
    SELECT number FROM t1, ft WHERE t1.number=ft.rowid ORDER BY date
  </verbatim>

  SQLite currently does a linear scan of table ft, joins it using rowid-lookup against t1, then sorts the results in a temporary b-tree. This is inefficient.

  <hr><i>dan added on 2010-03-27 09:11:20:</i><br>

  The problem is that when assessing the cost of using table "ft" as the outer loop, the query planner is not including the extra cost of sorting the results in-memory. This cost is considered in the first query when assessing the costs of using "bt" as the outer loop.

  If the extra cost associated with sorting is included, then the planner concludes that it is better to use the index on column "date" as the outer loop.


tkt_uuid: f3162063fd75dba32f0b919581c0d62ed7edbb3b
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.6.23
private_contact: b1d53ef4b1b67597b3e6a0bca6c27f9e758092ef
resolution: Fixed
title: Crash compiling "INSERT INTO tbl DEFAULT VALUES" if tbl has BEFORE or INSTEAD OF triggers.
comment:
  <verbatim>
  SQLite version 3.6.23
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a, b); 
  sqlite> CREATE TRIGGER tt1 BEFORE INSERT ON t1 BEGIN SELECT 1; END;
  sqlite> INSERT INTO t1 DEFAULT VALUES;
  Segmentation fault (core dumped)
  </verbatim>

  <hr><i>drh added on 2010-03-30 12:34:40:</i><br>
  This problem appears to have been present in the code ever since the
  INSERT INTO ... DEFAULT VALUES construct was added in
  2006 (version 3.3.8) with check-in [bc84cb54b0df]


tkt_uuid: 883034dcb54218347609fa340c1a725ae0de7145
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.23
private_contact: 77eecbbd6d208b172a571f0340526ba80a2b2b50
resolution: Fixed
title: Column cache problem with aggregate queries
comment:
  A bug in the way the column-cache is used by aggregate queries means that if a column that is part of an argument expression to an aggregate function is also a result of the SELECT statement, its register may be overwritten by another value between the time when it is passed to the aggregate and when it is returned by itself.

  For example:

  <verbatim>
  SQLite version 3.6.23
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t2(a PRIMARY KEY, b);
  sqlite> INSERT INTO t2 VALUES('abc', 'xxx');
  sqlite> INSERT INTO t2 VALUES('def', 'yyy');
  sqlite> SELECT a, max(b || a) FROM t2 WHERE (b||b||b)!='value' GROUP BY a;
  yyyyyyyyy|xxxabc
  def|yyydef
  sqlite> SELECT b, max(a || b) FROM t2 WHERE (b||b||b)!='value' GROUP BY a;
  yyyyyyyyy|abcxxx
  yyy|defyyy
  </verbatim>

  <hr><i>dan added on 2010-03-31 15:10:36:</i><br>

  Bug has been present since at least 3.6.14.


  <hr><i>dan added on 2010-03-31 15:11:24:</i><br>

  Bug has been present since at least 3.6.14.


tkt_uuid: 30349e0f68384eaeb622c8abb93a20b1eb214997
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Cosmetic
foundin: 3.6.23.1
resolution: Open
title: Yet more FTS3 symbols need to be made private
comment:
  fts3DbExec and fts3SnippetShift need to be static or SQLITE_PRIVATE

  Instead of me repeatedly having to point this out over the years, please add something to the release procedure to catch this kind of thing.  It is trivial to automate.

      nm --extern-only --defined-only sqlite.o | egrep -v ' (__bss_start|_edata|_end|_fini|_init)$'

  Compile with SQLITE_API and SQLITE_EXTERN defined to be static

  <hr><i>rogerb added on 2010-07-19 05:00:53:</i><br>
  This has now been fixed.


tkt_uuid: 9d68c883132c8e9ffcd5b0c148c990807b5df1b7
type: Code_Defect
status: Fixed
subsystem: Pager
priority: Immediate
severity: Critical
resolution: Fixed
title: Database corruption following OOM fault
comment:
  If auto_vacuum in enabled and the page size of the database is less than
  the media sector size and if an OOM error occurs at just the right moment
  while pages are being shuffled around during a DROP TABLE and then a
  ROLLBACK occurs, the pager
  might lose track of which pages need to be rolled back and then fail
  to roll back one or more pages that were modified.  This will leave the
  database in an inconsistent and corrupted state.


tkt_uuid: 02a8e81d4457d3627ad49b5e0adbce9d1ecdd38e
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.23
private_contact: 08eb919acd3da9afdb9bab627f6f672426a98cd9
resolution: Fixed
title: LIMIT clause on sub-select in FROM clause of a SELECT in a UNION ALL interpreted incorrectly
comment:
  <verbatim>
  dan@computer2:~/work/sqliteG/bld$ ./sqlite3 
  SQLite version 3.6.23
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> create table t1(a);
  sqlite> insert into t1 values(1);
  sqlite> insert into t1 values(2);
  sqlite> select * from (select * from t1 LIMIT 1) UNION ALL select 3;
  1
  sqlite> select * from (select * from t1 LIMIT 1) UNION select 3;
  1
  3
  </verbatim>

  <hr><i>drh added on 2010-04-15 14:43:49:</i><br>
  The problem was introduced by check-in [edca8913ca012fc0c17343a27f819de95147b1bd]
  on [2005-10-06].


tkt_uuid: bb1977f6e9f23108b6b02ddc79817904b907f45b
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.6.23.1
resolution: Rejected
title: Concatenation can create very invalid strings
comment:
  With a UTF16 encoded database:

  <pre>
      sqlite> insert into foo values('CD' || x'00' || 'E');
      sqlite> select cast(x as blob) from foo;
      INSERT INTO "table" VALUES(X'43004400004500');
      sqlite> select typeof(x) from foo;
      text
      sqlite> select length(x) from foo;
      4
  </pre>

  Note that the string is 7 bytes long which is very invalid for UTF16 encoding.  It is unlikely all code paths have been tested to ensure they cope with this correctly, don't read the 8th byte, deal with potential issues such as introducing surrogate pairs this way that are not valid etc.

  Suggestions include making || with blobs on either side result in a blob, more careful blob to string coercion that always results in a valid string, giving an error etc.

  <hr><i>drh added on 2010-04-22 12:08:03:</i><br>
  After a due consideration, the developers are unanimous that the behavior of strings and blobs
  and the concatenate operator should remain unchanged.  With over a billion deployments, a
  behavior change like the one recommended here is too likely to break something somewhere.

  One might argue that the suggested behavior makes more sense.  But we could argue back:
  Garbage in, garbage out.

  This ticket does, however, point out the desirability of very carefully testing the behavior when
  UTF16 strings have an odd number of bytes - to verify that there are no buffer overrun bugs
  elsewhere in the code.  We'll be undertaking those tests soon.


tkt_uuid: c39ff61c43bb72660b776df7be505671852df5be
type: Code_Defect
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Critical
foundin: 3.6.23.1
private_contact: 94dfef9000c47d0c206718c346fe8dca734d8c15
resolution: Fixed
title: Erroneous FK constraint failure when reusing statement
comment:
  If a call to sqlite3_step() fails due to an immediate FK violation, after resetting the statement it continues to fail with SQLITE_CONSTRAINT, even if the bound parameters or database contents have changed so that it should succeed.

  The problem is that the "immediate FK constraint counter" associated with the statement handle is not being reset to 0 in sqlite3_reset().

  The attached C program demonstrates the problem.


tkt_uuid: 3dbdcdb14e7f41714a9678326d71962d19a538e8
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.23
resolution: Fixed
title: Assertion fault using indices with redundant columns
comment:
  The following script causes an assertion fault:

  <blockquote><pre>
  CREATE TABLE t(a,b);
  CREATE INDEX i ON t(a,b,a);
  SELECT * FROM t WHERE a=0 AND b=0;
  </pre></blockquote>

  The problem was introduced by check-in [c30cbba9ead1b4] on
  [2005-07-21].


tkt_uuid: 26ff0c82d1e905657fb20753f65c93e6e7aecd33
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.12
resolution: Fixed
title: Incorrect result when using unnamed "?" bindings
comment:
  The following SQL with integer bindings against parameters 1 and 2
  generates an incorrect result:

     SELECT CASE WHEN 1 THEN 99 ELSE ? END + ?;

  This problem was introduced by the OP_Variable optimization of
  check-in [48b77b04935d894] on [2009-02-20] for SQLite version 3.6.12.


tkt_uuid: 51914f6acd2cb462ede8e7272a5c23eb34a7a991
type: Performance_Issue
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.6.23.1
resolution: Fixed
title: nRef and owner should be #ifdef SQLITE_DEBUG
comment:
  The nRef and owner fields for SQLITE_MUTEX_PTHREADS should be inside #ifdef SQLITE_DEBUG since they are only used in assertion checking mutexes are held/not held.

  Currently a normal (non-debug) build will be pointlessly updating their values and calling pthread_self on every mutex acquisition and release.

  Additionally when they are declared they need to be declared as volatile as the held checking can give the wrong answer depending on the order they are updated by the compiler.

  <hr><i>rogerb added on 2010-05-13 23:00:39:</i><br>
  The original reporter confirms that adding volatile fixes the assertion failures they were previously able to experience.


tkt_uuid: dd08e5a988d00decc4a543daa8dbbfab9c577ad8
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.6.23
resolution: Fixed
title: Foreign key constraint fails to prevent consistency error.
comment:
  The final DELETE in the following SQL sequence should be disallowed by the
  FK constraint on table t2.  But it is allowed:

  <verbatim>
     PRAGMA foreign_keys=ON;
     CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
     CREATE UNIQUE INDEX i1 ON t1(a,b);
     INSERT INTO t1 VALUES(100,200);
     CREATE TABLE t2(w,x,y,FOREIGN KEY(x,y) REFERENCES t1(a,b));
     INSERT INTO t2 VALUES(300,100,200);
     DELETE FROM t1;
  </verbatim>


tkt_uuid: f973c7ac3101ef4f30748aef64578a7b29ba5a17
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.23.1
resolution: Fixed
title: Incorrect affinity for indexed range constraint
comment:
  The query below gets the correct answer before the index is created.  But after
  the index is created, the wrong answer is returned.

  <blockquote><pre>
  CREATE TABLE t1(a INT, b INT);
  INSERT INTO t1 VALUES(5,5);
  SELECT * FROM t1 WHERE a=5 AND b>0 AND b<'4';
  CREATE INDEX i1 ON t1(a,b);
  SELECT * FROM t1 WHERE a=5 AND b>0 AND b<'4';
  </pre></blockquote>

  The problem seems to have been introduced by check-in
  [ebb0c8a3e977dc741704e733b5a5d931d9b27028] on [2009-11-17].
  This check-in was part of release 3.6.21.


tkt_uuid: fc62af45239916f7605ba07401b6f975d671bbdc
type: Code_Defect
status: Fixed
subsystem: Pager
priority: Immediate
severity: Critical
foundin: No released versions.
resolution: Fixed
title: Executing "PRAGMA journal_mode" may delete journal file while it is in use.
comment:
  If "PRAGMA journal_mode" is used to change a connection from TRUNCATE or PERSIST mode to WAL, MEMORY, OFF or DELETE mode, the journal file is deleted. Even if it is currently being used by another connection (possibly in another process).

  On unix, if that other connection (possibly in another process) is executing a write transaction when this happens, and the process exits unexpectedly for some reason, there will be no hot-journal file left in the file-system. If the connection had begun writing to the database file before the unexpected exit, the database may be left in a corrupt state.

  <hr><i>dan added on 2010-06-17 09:52:50:</i><br>
  Introduced in commit [b78e58ae15] on [2010-03-19 15:48:14].

  <hr><i>dan added on 2010-06-17 10:44:33:</i><br>
  The fix checked in for this ([1ec74591a9]) is sub-optimal in the sense that if a writer changes from "PRAGMA journal_mode=PERSIST" to "PRAGMA journal_mode=MEMORY", the journal file will never be removed from the file-system. It is not hot (as the first part of it is zeroed), but SQLite will be forced to open and close the file at the start of every read or write transaction to verify this.


tkt_uuid: d11f09d36e7cb0821e01f47278aa58a503423be6
type: Code_Defect
status: Closed
subsystem: Pager
priority: Immediate
severity: Critical
foundin: 3.6.23.1
resolution: Fixed
title: Cache corruption caused by ROLLBACK with synchronous=NORMAL
comment:
  If a very large transaction is rolled back in synchronous=NORMAL mode (so large that a cache-spill has occurred before the ROLLBACK happens), the in-memory page cache can become corrupted. If the database connection then writes to the database, it is possible that the database can become corrupted too.

  <hr><i>dan added on 2010-06-26 16:17:33:</i><br>
  Introduced (or possibly just exposed) in [0906597698b].


tkt_uuid: 80e031a00f45dca877ed92b225209cfa09280f4f
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.6.23.1
resolution: Fixed
title: Result of IN should never be NULL if the RHS is empty
comment:
  When the right-hand side (RHS) of an IN or NOT IN operator is an empty set,
  then the result should always be false or true, respectively, even if the
  left hand operand is NULL.  The current (incorrect) behavior of SQLite is 
  to always yield a NULL result if the left operand is NULL.


tkt_uuid: f817a4575241cf9beda96f6c8b8e6d181accaa43
type: Feature_Request
status: Closed
subsystem: VFS
priority: Immediate
severity: Minor
foundin: 3.7.0
resolution: Overcome_By_Events
title: CantOpen doesn't include name of file that can't be opened
comment:
  If opening a database fails, all you get as the error text is "unable to open database file".

  Nowadays opening a file is rather complicated since the name is run through xFullPathname and there are several ancilliary files that are also used (-journal, -shm, -wal etc).

  This makes it very hard for the users to work out which file exactly the problem was with.


tkt_uuid: e090183531fc274747ee47477d793857f4f76b71
type: Feature_Request
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.6.23.2
private_contact: 22d2ebb9a6dbfa47bd4b558ba855d0331d912298
resolution: Fixed
title: like should use index when pattern has no wildcard
comment:
  As discussed by Dan Kennedy and Ben Trumbull in a recent mail thread:

  It's arguably a defect. In this case the pattern used
  with LIKE must contain at least one wildcard character.
  So "value LIKE 'abc'" will not use the index, but
  "value LIKE 'abc%'" should.

  On Jul 21, 2010, at 1:59 PM, Ben Trumbull wrote:

  Hi Richard, Dan,

  We're having trouble getting section 4.0 like optimization in <http://www.sqlite.org/optoverview.html> to work as described. If I have:

  CREATE TABLE ABMultiValue (UID INTEGER PRIMARY KEY, record_id INTEGER, property INTEGER, identifier INTEGER, label INTEGER, value TEXT);
  CREATE INDEX testLike on ABMultiValue(value COLLATE NOCASE);

  and then compare = with LIKE (both collate nocase) I just can't get the LIKE expression to use the index:


tkt_uuid: 4711020446da7d93d993aa22d563e8de6cb65799
type: Feature_Request
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.6.23.2
private_contact: 22d2ebb9a6dbfa47bd4b558ba855d0331d912298
resolution: Fixed
title: should use collate nocase index even if column isn't declared nocase
comment:
  As (also) discussed by Dan Kennedy and Ben Trumbull:

  A query will not use a collate nocase index when the column isn't also declared collate nocase:

  I think the column (not just the index) has to be declared
  with COLLATE NOCASE. As per condition (6) here:

   http://www.sqlite.org/optoverview.html#like_opt

  I'm not sure why that is at the moment. It does seem like a
  NOCASE index on a column with a different default collation
  type would work just as well.

  Regards,
  Dan.


tkt_uuid: 5e0423b058fa5adf8756391a04b047230d03fba0
type: Documentation
status: Fixed
subsystem: VFS
priority: Immediate
severity: Important
foundin: 3.7.0
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: xAccess behaviour is not documented, changed in Windows 3.7.0
comment:
  The documentation doesn't really define what xAccess should do, in particular when should it return False versus returning an error.

    http://sqlite.org/c3ref/vfs.html

  Under Unix you cannot have invalid filenames, except with some specialised non-default file systems.  However Windows does have quite a few restrictions.  For example it doesn't permit double quotes, angle brackets, asterisk, question mark etc in file names.  (Note that colon is permitted to name an alternate data stream.)

  In my testing I use a filename of "<bad<filename:" and then call xAccess.  Prior to SQLite 3.7.0 the Windows VFS would return False.  With 3.7.0 it now returns SQLITE_IOERR with the helpful error text of "error".

  <hr><i>rogerb added on 2010-07-22 06:11:59:</i><br>
  The Windows VFS also definitely appears to contradict the limited description in the documentation.  In particular calling xAccess with a filename of "." (ie currently directory) and a flag of SQLITE_ACCESS_EXISTS returns False.  It used to return True prior to SQLite 3.7.0.


tkt_uuid: 8011086c85c6c404014c947fcf3eb9f42b184a0d
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.0
resolution: Fixed
title: Query planner creates unnecessary and expensive automatic index
comment:
  Consider the following SQL:

  <blockquote><pre>
  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
  CREATE TABLE t2(x INTEGER PRIMARY KEY, y);
  SELECT b FROM t1
   WHERE t1.a IN (SELECT x FROM t2 WHERE y=?);
  </pre></blockquote>

  The query planner is creating an automatic index for the T2.Y column even
  though a simple linear scan of T2 would be a less costly plan.

  This problem was introduced in version 3.7.0, which is the first version
  of SQLite to support automatic indices.  The work-around is to use

  <blockquote><pre>
  PRAGMA automatic_index=OFF;
  </pre></blockquote>

  to disable the automatic index mechanism, or to compile SQLite using
  the -DSQLITE_OMIT_AUTOMATIC_INDEX compile-time option.


tkt_uuid: ce7c133ea6cc9ccdc1a60d80441f80b6180f5eba
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.0
resolution: Fixed
title: Foreign key constraint fails when it should succeed.
comment:
  Consider the following SQL:

  <blockquote><pre>
    PRAGMA foreign_keys=ON;
    CREATE TABLE t1(a INTEGER PRIMARY KEY, b INTEGER);
    CREATE UNIQUE INDEX i1 ON t1(a,b);
    INSERT INTO t1 VALUES(100,200);
    CREATE TABLE t2(w INTEGER,x INTEGER,y INTEGER,FOREIGN KEY(x,y) REFERENCES t1(a,b));
    INSERT INTO t2 VALUES(300,100,200);
    UPDATE t1 set b = 200 where a = 100;
  </pre></blockquote>

  The final update fails, even though it should not.

  This problem seems to have been introduced by 
  checkin [636f86095eb1f4bdcfb0c9ed846c4c6b3589c10b] on [2010-05-14 19:24:02]
  which was to
  fix prior ticket [dd08e5a988d00decc4a543daa8dbbfab9c577ad8].
  It appears that the fix to the prior problem was not quite right.

  The new problem first appears in the 3.7.0 release.

  <hr><i>drh added on 2010-07-29 01:55:02:</i><br>
  The issue was in the handling of multi-column foreign key constraints
  where one of the columns was an integer primary key.  The error has
  always been in the code.  It was simply masked by the prior error 
  of ticket [dd08e5a988d00decc4a543daa8dbbfab9c577ad8].  The fix in
  check-in [636f86095eb1f4bdcfb0c9ed846c4c6b3589c10b] is correct.  That
  fix did not introduce a new problem; it merely brought an existing
  problem into view.

  The Severity of this ticket is downgraded to Important since one
  can easily argue that any schema that has a multi-column
  foreign key constraint where one of the columns is a primary key
  is a broken schema.  Even so, we ought to handle even broken
  schemas correctly, which should be the case after check-in
  [53902f7d4a46aa70ecc5bf180a01ff888d52686a].

  <hr><i>anonymous added on 2010-07-31 06:43:18:</i><br>
  It seems that foreign key isnt't working as explained on http://www.sqlite.org/foreignkeys.html everything there succeed when it shouldn't, I tested with 3.6.23.1 with the same results.
  I even downloaded a windows binary of sqlite3.exe 3.7 from here to see if the problem was on my compilation flags but it isn't because the binary provided here also behave the same way (it parses without complain but work as if no foreign key restrictions exist).

  <hr><i>anonymous added on 2010-07-31 06:49:02:</i><br>
  Sorry but it was my fault I didn't noticed the "PRAGMA foreign_keys" I was expecting it should be "ON" by default but it seems that is the opposite, maybe for backward compatibility. 
  I'm not sure if this is the best default settings ?
  PRAGMA foreign_keys=OFF;

  <hr><i>anonymous added on 2010-07-31 06:57:00:</i><br>
  Why not have a compiler flag to set the default for "PRAGMA foreign_keys" this way new development can have this pragma "ON" by default ?


tkt_uuid: c43940c49b74c70a691dfe35c7329399a0969793
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Severe
foundin: 3.7.0
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: sqlite3_profile gives milliseconds (was nanoseconds in earlier releases)
comment:
  Prior to SQLite 3.7.0 the sqlite3_profile callback gave the timings in nanoseconds.  In 3.7.0 as a side effect of adding the xCurrentTimeInt64 VFS, timings are now returned in milliseconds.  This undocumented change breaks profilers such as the one I include with my Python wrapper.

  Some additional notes:

    *  The documentation doesn't actually state the units
    *  In the code sqlite3_profile() is marked with SQLITE_EXPERIMENTAL
    *  In the documentation sqlite3_profile() is not marked as experimental


tkt_uuid: f3e5abed55378ccb11396096c25ca338b295a46c
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.0 (likely all versions)
resolution: Fixed
title: Problems retrying COMMIT after SQLITE_BUSY on multi-file transaction
comment:
  In SQLite, an attempt to COMMIT a transaction in rollback-journal mode may fail with an SQLITE_BUSY error. The application may retry the COMMIT later on, and it may succeed.

  If this happens when committing a multi-file transaction, each attempt to COMMIT creates a new master-journal file in the file-system. Only the master-journal file associated with the final, successful, COMMIT is deleted. Leaked master-journals are left in the file-system.

  Also, at least one of the journal files involved in the transaction contains a master-journal pointer that points to a master-journal other than the last one created. This means the final COMMIT is not atomic across all database files. A particularly unlucky process or OS failure may leave the writes to a subset of the databases committed.


tkt_uuid: 168d0f7176ab76067736c3f25bad9aa7248683a0
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.0
resolution: Fixed
title: Segfault while reading database with a corrupt index
comment:
  If a database file contains indices which are out-of-date with their tables
  (in other words, if the database file is corrupt) then it is possible to
  segfault while doing a SELECT on that database.

  <hr><i>drh added on 2010-08-01 22:25:47:</i><br>
  Fixed by checkin [83395a3d24f18170fc068d9e644291678694c3f0]


tkt_uuid: c38baa3d969eab7946dc50ba9d9b4f0057a19437
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.0
resolution: Fixed
title: REPLACE on a CHECK constraint does not work as advertised
comment:
  The requirements say that the ON CONFLICT REPLACE conflict resolution 
  algorithm should work like ABORT when a CHECK constraint fails.  But
  the implementation actually works like ROLLBACK.  The following code
  demonstrates:

  <blockquote><pre>
  CREATE TABLE t1(a CHECK(a!=2));
  BEGIN;
  REPLACE INTO t1 VALUES(4);
  REPLACE INTO t1 VALUES(2);
  REPLACE INTO t1 VALUES(3);
  COMMIT;
  SELECT * FROM t1;
  </pre></blockquote>

  The problem appears to have been introduced by check-in
  [b213614abff90c65cc7228c87f4973150e8a9679] on [2007-07-23 19:39:47].


tkt_uuid: 51ae9cad317a1b609735f99c88168bbf3661feb7
type: Code_Defect
status: Fixed
subsystem: B-Tree
priority: Immediate
severity: Critical
foundin: 3.7.0
resolution: Fixed
title: Database corruption by alternating writes from 3.7.0 and 3.6.23
comment:
  If the same database file is alternately written by SQLite version 3.7.0
  and version 3.6.23.1 (or earlier), database corruption can occur.  The
  failure scenario is this:

    1.  The database is created and initially populated using version 3.7.0.
    2.  Version 3.6.23.1 adds content, causing the size of the database to increase.
    3.  Version 3.7.0 updates the database without increasing its size

  The corrupt is that the in-header-size field at offset 28 into
  the header ends up being set incorrectly.  This field was always zero for
  SQLite versions 3.6.23.1 and earlier; it was introduced for the first time
  in version 3.7.0.  Version 3.7.0 knows to only use the in-header-size if
  the change-counter at offset 24 matches the change-counter-copy at 
  offset 92.  And so
  normally, 3.6.23.1 and 3.7.0 can interact without problems.  Step 2 in the
  failure scenario above causes the in-header-size field to be set incorrectly
  (because 3.6.23.1 changes the size of the database but does not update the
  in-header-size field).  The database is still valid after step 2, though,
  since the change-counter-copy at offset 92 no longer matches 
  the change-counter at offset
  28.  However, step 3 updates both the change-counter and 
  change-counter-copy
  without correcting the in-header-size, resulting in database corruption.

  <hr><i>dan added on 2010-08-04 11:39:07:</i><br>
  Fixed by commit [65b8636ac6].


tkt_uuid: 13f033c865f878953fb0412b389dd1d1143d1dce
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.0
resolution: Fixed
title: Performance regression
comment:
  Consider the following SQL:

  <blockquote><pre>
  CREATE TABLE t1(a INTEGER PRIMARY KEY,b,c);
  CREATE INDEX t1c ON t1(c);
  INSERT INTO t1 VALUES(1,2,3);
  CREATE TABLE t2(x, y);
  ANALYZE;
  UPDATE sqlite_stat1 SET stat='20000 100';
  explain query plan
  SELECT * FROM t2, t1 WHERE t2.x=5 AND t1.a=t2.y;
  </pre></blockquote>

  The SELECT at the end should ideally do a full table scan over table T2
  and then use the PRIMARY KEY to look up entries of T1, resulting in
  an O(NlogN) execution time.  However, beginning with SQLite version 3.7.0
  (and specifically check-in [defaf0d99a807] on [2010-04-15 01:04:54], 
  the query planner does two full table scans on T1 first and then T2 as
  the inner loop, resulting an O(N**2) execution time.


tkt_uuid: e4b8a2ba6e06d34086362c0caeff05e523184a43
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.1
resolution: Fixed
title: Problem with DISTINCT in sub-queries in a FROM clause.
comment:
  Internally, SQLite is transforming queries of the form:

    SELECT f(x,y) FROM (SELECT DISTINCT x, y FROM tbl);

  to

    SELECT DISTINCT f(x,y) FROM tbl;

  But this is only valid if f(x,y) is distinct for all distinct (x,y) rows. And if f(x,y) is really just a function of (x,y). It may not be if it includes some user-defined SQL functions, such as randomblob().

  <hr><i>dan added on 2010-08-13 16:50:01:</i><br>

  This problem has been present since at least version 3.5.1.


tkt_uuid: 5e10420e8de26a086cd0f532643bedbfa94921e0
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.1
resolution: Fixed
title: Free-list corruption in incremental-vacuum mode
comment:
  Running an incremental vacuum can result in free-list corruption in an obscure case. The case is hit if the incremental vacuum code attempts to extract the second of two empty free-list trunk pages (trunk pages with no references to free-list leaves), and the page extracted is not the last trunk page in the free-list.

  Running integrity-check after this occurs produces output similar to:

  <verbatim>
  *** in database main ***
  Main freelist: invalid page number 258
  Page 6 is never used
  Page 7 is never used
  Page 8 is never used
  Page 9 is never used
  Page 10 is never used
  Page 11 is never used
  Page 12 is never used
  Page 13 is never used
  ...
  </verbatim>

  This problem first appeared in 3.6.16.

  <hr><i>drh added on 2010-08-23 17:08:42:</i><br>
  Fixed by check-in [255f1eefa373153942c67b18b22177933657911d]


tkt_uuid: 313723c356483eff2a4c4bdd2c66ad342da7bd6a
type: Code_Defect
status: Fixed
subsystem: Pager
priority: Immediate
severity: Minor
foundin: 3.7.2
resolution: Fixed
title: Database corruption following an I/O error on xShmLock
comment:
  If the xShmLock() method of the VFS reports an I/O error, then SQLite might
  fail to clear its cache properly under some circumstances.  This could (in
  theory) lead to a corrupt database file.

  This problem is more theoretical than real since we are not aware of any
  real VFS where it is possible to get an I/O error on the xShmLock() method.
  On unix and windows, the xShmLock() method might fail to get the lock, but
  it does not generate an I/O error unless the host computer is very, very
  sick.  And even if you did get an I/O error on on xShmLock(), it is still 
  very tricky to get into a situation that corrupts the cache.

  This problem was discovered while running some obscure TH3 tests that use
  a modified VFS that simulates multiple I/O errors on xShmLock() in rapid
  succession.

  This problem is so obscure that one is tempted to not write up a bug report
  on it.  But our policy is to always write a bug report for problems that can
  result in database corruption, so here it is...


tkt_uuid: b351d95f9cd5ef17e9d9dbae18f5ca8611190001
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.7.2
resolution: Fixed
title: Bug in "INSERT INTO ... SELECT" statements that use CASE
comment:
  <verbatim>
  SQLite version 3.7.2
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a, b);
  sqlite> CREATE TABLE t2(x);
  sqlite> INSERT INTO t1 VALUES(1, 'ENG');
  sqlite> SELECT CASE b WHEN 'UND' THEN NULL ELSE b END FROM t1;
  ENG
  sqlite> INSERT INTO t2 SELECT CASE b WHEN 'UND' THEN NULL ELSE b END FROM t1;
  sqlite> SELECT x FROM t2;

  </verbatim>

  <hr><i>drh added on 2010-09-28 07:21:50:</i><br>
  This problem results from misuse of the OP_SCopy (shallow copy) opcode.
  The error sequence is as follows:

    1.  OP_SCopy A B
    2.  change or destroy A
    3.  attempt to use B

  The change to A in step 2 corrupts the shallow copy in B.

  The specific manifestation of the problem described by this ticket was
  introduced in version 3.6.0 (2008-07-16) but the larger SCopy misuse
  problem probably goes back much further.

  The problem is very obscure, as evidenced by the fact that nobody has
  hit it in over two years of heavy use.  But the problem could
  potentially strike any application.  All applications are encouraged
  to upgrade to the next release that fixes this problem.

  In addition to fixing the specific manifestation of the problem described
  by this ticket, new assert() statements have been added which are designed
  to detect similar problems elsewhere in the VDBE code generator.


tkt_uuid: 8454a207b9fd2243c4c6b7a73f67ea0315717c1a
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.2
resolution: Fixed
title: ALTER TABLE ADD COLUMN with negative DEFAULT value
comment:
  A negative default value on an added text column drops the negative sign.
  Here is an example:

  <blockquote><pre>
  CREATE TABLE t1(a);
  INSERT INTO t1 VALUES(1);
  ALTER TABLE t1 ADD COLUMN b TEXT DEFAULT -123.5;
  SELECT b FROM t1;
  </pre></blockquote>

  The result of the SQL above should be -123.5 but it is coming out as a
  positive 123.5.

  This problem appears to have always existed, ever since ALTER TABLE ADD COLUMN
  was first introduced in SQLite version 3.2.0 on [2005-03-21].

  The problem is not considered to be critical, however, since it
  is questionable why anybody would want to put a numeric default value
  on a text column.


tkt_uuid: 3998683a16a7076e08f5585c1f4816414c8c653a
type: Code_Defect
status: Fixed
subsystem: Utilities
priority: Immediate
severity: Severe
foundin: 3.7.2
resolution: Fixed
title: Some valid floating-point literals are not recognized.
comment:
  In some contexts, floating point constants where the mantissa begins or ends
  with a decimal point, the constant is not recognized as a valid floating-point
  number. For example:

  <blockquote><pre>
  CREATE TABLE t1(x, y REAL);
  INSERT INTO t1 VALUES(1, '1.0');
  INSERT INTO t1 VALUES(2, '.125');
  INSERT INTO t1 VALUES(3, '123.');
  INSERT INTO t1 VALUES(4, '123.e+2');
  INSERT INTO t1 VALUES(5, '.125e+3');
  INSERT INTO t1 VALUES(6, '123e4');
  SELECT x FROM t1 WHERE typeof(y)=='real';
  </pre></blockquote>

  The output from the query on the last line of the SQL above should be
  size integers, 1 through 6.  But in all SQLite versions going back
  to 3.1.3 (the oldest version tested), only the first and last integers
  are output.  The affinity transformation routines do not recognize
  entries 2 through 5 as floating point values because of the decimal
  point at the beginning or end of the mantissa.


tkt_uuid: 38cb5df375078d3f9711482d2a1615d09f6b3f33
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.2
resolution: Fixed
title: LIMIT ignored on compound query with subqueries
comment:
  In the following code, multiple rows of output are generated in spite of the
  LIMIT 1 clause:

  <blockquote><pre>
  CREATE TABLE t(a);
  INSERT INTO t VALUES(1);
  INSERT INTO t VALUES(2);

  SELECT * FROM (SELECT * FROM t ORDER BY a)
  UNION ALL SELECT * FROM (SELECT a FROM t)
  LIMIT 1;
  </pre></blockquote>

  This defect has been in the code going back to version 3.1.3 ([2005-02-19])
  and probably before that.


tkt_uuid: a04e42a3fcacaffa3133436fda957343e2be0fee
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.7.3
resolution: Fixed
title: Memory leak in sqlite3_create_collation_v2()
comment:
  If an OOM error occurs during sqlite3_create_collation_v2(), the function 
  still returns SQLITE_OK instead of returning SQLITE_NOMEM.  And the
  destructor for the application data is not called.


tkt_uuid: 860399cc408f2dd5f41aed44bc4d3296f2755ef9
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.7.3
resolution: Fixed
title: Potential memory leak in sqlite3_bind_text()
comment:
  In the following C interfaces:

    *  sqlite3_bind_blob()
    *  sqlite3_bind_text()
    *  sqlite3_bind_text16()

  SQLite promises to take responsibility for invoking the destructor on the
  bound string or blob once that string or blob is no longer needed. However,
  this sometimes does not happen if the bind interface call itself fails.
  For example, the destructor is called on an SQLITE_TOOBIG error but is
  omitted on an SQLITE_RANGE error.  The interfaces should be consistent
  and always invoke the destructor regardless of the error.


tkt_uuid: 80ba201079ea608071d22a57856b940ea3ac53ce
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.3
resolution: Fixed
title: Bug involving subqueries and the OR optimization
comment:
  SELECT statements can return incorrect results in certain cases where the following are true:

  <ul>
  <li> The query is a join,
  <li> The query takes advantage of the OR optimization,
  <li> At least one branch of the optimized OR expression in the WHERE clause involves a subquery.
  <li> At least one branch of the optimized OR expression in the WHERE clause refers to a column of a table other than the table to which the OR optimization applies.
  </ul>

  For example:

  <verbatim>
    CREATE TABLE t1(a);
    INSERT INTO t1 VALUES('A');
    CREATE INDEX i1 ON t1(a);

    CREATE TABLE t2(b);
    INSERT INTO t2 VALUES('B');

    CREATE TABLE t3(c);
    INSERT INTO t3 VALUES('C');

    SELECT * FROM t1, t2 WHERE
       (a='A' AND b='X')
    OR (a='A' AND EXISTS (SELECT * FROM t3 WHERE c='C'))
  </verbatim>

  The bug causes the query above returns zero rows instead of the expected result - one row.

  This bug was discovered in a real application and reported on the mailing list. Bisection reveals that it was introduced on 2009-08-13 by commit [86a06dd049]. It first appeared in release 3.6.18.


tkt_uuid: 5d863f876ee9561b95e279b3381450514b6da668
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.4
resolution: Fixed
title: DB corruption following rollback-WAL-VACUUM-rollback
comment:
  An obscure method for corrupting databases has been discovered, involving
  transitions between rollback journaling modes and WAL mode, with a VACUUM
  thrown in the middle.  The crux of the problem is that the change-counter
  field at offset 24 in the database file was being reset to zero by the
  sequence of commands:

  <pre>
      PRAGMA journal_mode=WAL;
      VACUUM;
      PRAGMA journal_mode=DELETE;
  </pre>

  The change-counter is used by database connections in rollback mode to
  detect when other database connections have modified the database, so
  that they can reset their caches.  (Other mechanisms are used in WAL mode.)

  So if the change counter is initially zero (perhaps due to having previously
  run the command sequence shown above) and two database connections
  are hooked to the database in rollback mode and one connection modifies
  the database and then does the sequence of commands above, 
  the other database connection will not notice the change and will hence
  perform subsequent operations using a stale cache.  This can lead to
  database corruption.

  This malfunction was discovered during testing and has not been observed
  in the wild.


tkt_uuid: 9013e13dba5b58c7d03f12c60424826b7214e1ea
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.5
resolution: Fixed
title: ATTACH authorization fails when database name is not a literal
comment:
  The authorizer ([http://www.sqlite.org/c3ref/set_authorizer.html]) is called
  to validate every ATTACH command.  The third parameter to the authorizer
  callback should be the name of the database file that is to be attached.
  But if the database name is specified as a parameter:

  <blockquote><pre>
  ATTACH $dbname AS newdb;
  </pre></blockquote>

  Then the name of the parameter is sent as the 3rd parameter, not the name
  of the file.  Or, if the database name is an arbitrary expression, the
  3rd parameter is undefined.

  The probable fix is to document that whenever anything other than a
  string literal is used in an ATTACH statement that the 3rd argument to
  authorizer callback is NULL.  In other words, a NULL filename to the
  SQLITE_ATTACH authorizer signifies that the filename is unknown at
  compile-time.

  The code needs to be changed to implement the above, and the documentation
  needs to be changed to explain that this is how it works.


tkt_uuid: 89b8c9ac540eb58af7c79f1b4995a44fa2d8fa59
type: Code_Defect
status: Fixed
subsystem: B-Tree
priority: Immediate
severity: Minor
foundin: 3.7.0 - 3.7.5
resolution: Fixed
title: False reports of corruption after writing db with two different versions (3.7 and 3.6).
comment:
  Version 3.7.0 and greater may incorrectly report that an auto-vacuum capable database is corrupt if the following sequence occurs:

    #  The database is written by version 3.7.0 or greater, then
    #  The database file is written to by a version earlier than 3.7.0 and the write operation causes the database file to shrink (either because data was removed from the db in auto-vacuum mode or because "PRAGMA incremental_vacuum" was executed in incr-vacuum mode).

  This leaves the database in a state where 3.7.0 and greater report that it is corrupt (and no data can be read from it). It is not actually corrupt, and all data can still be read by earlier versions.

  A database can be "repaired" by vacuuming it with a version earlier than 3.7.0.

  <verbatim>
     SQLite version 3.7.6
     Enter ".help" for instructions
     Enter SQL statements terminated with a ";"
     sqlite> PRAGMA auto_vacuum = 1;
     sqlite> CREATE TABLE t1(x);
     sqlite> .quit

     SQLite version 3.6.23.1
     Enter ".help" for instructions
     Enter SQL statements terminated with a ";"
     sqlite> DROP TABLE t1;
     sqlite> .quit

     SQLite version 3.7.6
     Enter ".help" for instructions
     Enter SQL statements terminated with a ";"
     sqlite> PRAGMA integrity_check;
     Error: database disk image is malformed
  </verbatim>

  <hr><i>drh added on 2011-02-09 18:39:32:</i><br>
  The database can also be "repaired" in SQLite version 3.7.5 and later by using

     PRAGMA writable_schema=ON;

  Immediately after opening the connection, and then running VACUUM (or any other
  write operation) on the database.


tkt_uuid: 77aa3b1e6592582e38605d36733b35d51ae5e973
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.5
resolution: Fixed
title: Segfault on automatic index using unknown collating function
comment:
  If the query planner tries to create an automatic index that uses a column
  with an unknown collating function, an assertion fault or a segfault might occur.
  Suppose the schema is:

  <blockquote><pre>
  CREATE TABLE t1(x TEXT COLLATE unknown);
  </pre></blockquote>

  Then the following query (entered separately, from a shell that does not
  know about the "unknown" collating function) might fail:

  <blockquote><pre>
  SELECT * FROM t1 WHERE x IN ('abc','def');
  </pre></blockquote>


tkt_uuid: 752e1646fcc7b649184e49783577a7feb5f7fc9c
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
private_contact: 46134d5cec7e31a3cce01ff529b363274ceaf672
resolution: Open
title: Wrong result if DISTINCT used on subquery which uses ORDER BY.
comment:
  Wrong result if DISTINCT used on subquery which uses ORDER BY.

  <blockquote><pre>
  CREATE TABLE "test" ("letter" VARCHAR(1) PRIMARY KEY, "number" INTEGER NOT NULL);

  INSERT INTO "test" ("letter", "number") VALUES('b', 1);
  INSERT INTO "test" ("letter", "number") VALUES('a', 2);
  INSERT INTO "test" ("letter", "number") VALUES('c', 2);
  </pre></blockquote>

  Then the following query

  <blockquote><pre>
  SELECT "letter", "number" FROM "test" ORDER BY "letter", "number" LIMIT 1;
  </pre></blockquote>

  returns "a|2", the second row from the results as you would expect.
  However, if this query is used as the subquery of a SELECT DISTINCT

  <blockquote><pre>
  SELECT DISTINCT "number" FROM (SELECT "letter", "number" FROM "test" ORDER BY "letter", "number" LIMIT 1) AS "test";
  </pre></blockquote>

  then it returns "1" instead of the expected "2".

  If query optimizations are disabled, the SELECT DISTINCT returns "2".   

  This is most likely an issue with the query flattener.


tkt_uuid: b72787b1a7cea1fec8ab2f87682e2f370d8120d8
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Minor
foundin: 3.7.5
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: Expiry mechanism causes incorrect collateral damage
comment:
  The sqlite3ExpirePreparedStatements routine marks <b>all</b> statements as expired.  This includes statements that are not expired.

  Steps to reproduce:

    *  Prepare a statement (A)
    *  Alter the schema to invalidate cookie in A
    *  Prepare a statement (B)
    *  Run B and have A run as part of B
    *  A will find a bad cookie and cause *all* statements to be expired including the currently running B by calling sqlite3ExpirePreparedStatements
    *  When control returns to B it will then abort

  The bug is that sqlite3ExpirePreparedStatements expires all statements.  Note that B was prepared after the schema change and hence is perfectly valid and then is marked as expired while running.

  The original user report had B be "create table ... as select from ..." where the source table was a virtual table with A being an "insert" into an unrelated table.

  The abort comes from line 63626 of the 3.7.5 amalgamation, specifically this
  code at the top of the OP_OpenRead/OpenWrite case:
  <pre>
    if( p->expired ){
      rc = SQLITE_ABORT;
      break;
    }
  </pre>

  <hr><i>rogerb added on 2011-02-22 05:24:39:</i><br>
  Fix also confirmed using the original user's test case that involved virtual tables.


tkt_uuid: eebb8a2803c4ec0bb70f1ceaa90fe2281453437e
type: Build_Problem
status: Closed
subsystem: Other
priority: Immediate
severity: Important
foundin: 3.7.5
private_contact: 4fbe952a3fd0e91c08103e030a3f601770d643c0
resolution: Fixed
title: Inconsistent amalgamation zip archive file names
comment:
  In each of the minor releases 3.7.3, 3.7.4 and 3.7.5 the naming convention inside the amalgamation zip has changed.  For example 3.7.3 had no directories, 3.7.4 used the long integer version number as a directory and 3.7.5 uses a date.  Additionally the def file was dropped after 3.7.3.

  Although not immediately apparent the naming convention and contents are part of the interface for developers using SQLite.  Changing them should only happen as part of larger releases (eg going from 3.7 to 3.8) rather than in a minor change.
  <nowiki>
  <pre>
  Archive:  sqlite-amalgamation-3_7_3.zip
    Length      Date    Time    Name
  ---------  ---------- -----   ----
    4245940  2010-10-07 19:36   sqlite3.c
     291339  2010-10-07 19:37   sqlite3.h
      20686  2010-10-07 19:37   sqlite3ext.h
      82346  2010-10-07 19:37   shell.c
       3961  2010-10-07 19:37   sqlite3.def
  ---------                     -------
    4644272                     5 files
  Archive:  sqlite-amalgamation-3070400.zip
    Length      Date    Time    Name
  ---------  ---------- -----   ----
          0  2010-12-07 12:26   sqlite-amalgamation-3070400/
      82346  2010-12-07 12:26   sqlite-amalgamation-3070400/shell.c
    4326613  2010-12-07 12:26   sqlite-amalgamation-3070400/sqlite3.c
     295268  2010-12-07 12:26   sqlite-amalgamation-3070400/sqlite3.h
      23222  2010-12-07 12:26   sqlite-amalgamation-3070400/sqlite3ext.h
  ---------                     -------
    4727449                     5 files
  Archive:  sqlite-amalgamation-3070500.zip
    Length      Date    Time    Name
  ---------  ---------- -----   ----
          0  2011-01-31 07:31   sqlite-amalgamation-201101281703/
      83508  2011-01-31 07:31   sqlite-amalgamation-201101281703/shell.c
    4341715  2011-01-31 07:31   sqlite-amalgamation-201101281703/sqlite3.c
     298814  2011-01-31 07:31   sqlite-amalgamation-201101281703/sqlite3.h
      23222  2011-01-31 07:31   sqlite-amalgamation-201101281703/sqlite3ext.h
  ---------                     -------
    4747259                     5 files
  </pre>
  </nowiki>

  <hr><i>dan added on 2011-02-21 15:35:39:</i><br>

  The correct naming convention (and the one we will use going forward) is as for version 3.7.4 above. The 3.7.5 package on the website has now been fixed.

  <hr><i>rogerb added on 2011-02-21 18:41:30:</i><br>
  The .def file is still missing.  It was present until 3.7.3.


tkt_uuid: f7b4edece25c994857dc139207f55a53c8319fae
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.5
resolution: Fixed
title: sqlite3_update_hook gives incorrect table name with shared cache
comment:
  The fourth argument to the sqlite3_update_hook() callback is suppose to be
  the name of the table that is being updated.  However, this string pointer
  might point to deallocated memory if shared cache mode is in use.  Here
  is the scenario:

    1.  Two connections to the same database (call them A and B) in shared
        cache mode.
    2.  Connection A prepares a statement that will invoke the update hook.
    3.  Connection B makes a schema change but rolls back before committing.
    4.  Connection A evaluates the prepared statement.

  The prepared statement in step (2) contains a pointer to the table name
  string in the parsed schema.  Step (3) causes the schema to be reparsed,
  which invalidates that string.  But the rollback of step (3) means that
  the schema cookie is unchanged and so the prepared statement is not
  invalidated.  Step (4) then runs the prepared statement and passes the
  invalid string to the update hook callback.

  This problem was found by internal code review and has never been observed
  in the wild.


tkt_uuid: 83ea97620bd3101645138b7b0e71c12c5498fe3d
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
resolution: Fixed
title: ANALYZE on empty tables prevents the use of indices
comment:
  Running ANALYZE on a database that contains empty tables prevents any
  indices on the empty tables from being used even if content is subsequently
  added to the tables.

  The problem was originally reported on the sqlite-users mailing list.


tkt_uuid: bb3a86e890c8e96ab59af7acb43039b861387e0c
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.6
resolution: Fixed
title: SQLITE_FCNTL_SIZE_HINT overflows an array
comment:
  Due to an inadequately-tested last-minute change prior to the 3.7.6 release,
  use of the SQLITE_FCNTL_SIZE_HINT file control operation on unix builds where
  SQLITE_ENABLE_LOCKING_STYLE is 0 (or undefined) and HAVE_POSIX_FALLOCATE is
  defined will cause an array index to be off by one and will probably result
  in a segfault.

  The problem was introduced by check-in [51029d8430d2dbc] just prior to the
  release.

  The work-around is to omit the HAVE_POSIX_FALLOCATE compile-time option from
  the build.  Or, avoid using SQLITE_FCNTL_SIZE_HINT.


tkt_uuid: 6284df89debdfa61db8073e062908af0c9b6118e
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
resolution: Fixed
title: INSERT INTO ... SELECT bypasses foreign key constraint checks
comment:
  The INSERT statement marked in the SQL code below does not raise a
  foreign key constraint failure as it should:

  <blockquote><pre>
  PRAGMA foreign_keys=on;
  CREATE TABLE parent (p1 INTEGER PRIMARY KEY);
  CREATE TABLE child  (c1 INTEGER PRIMARY KEY REFERENCES parent(p1));
  CREATE TABLE source (s1 INTEGER PRIMARY KEY);
  INSERT INTO parent VALUES(137);
  INSERT INTO source VALUES(-476);
  INSERT INTO child SELECT * FROM source;  /* Should fail, but doesn't */
  SELECT * FROM child;
  </pre></blockquote>

  This issue was reported on the mailing list.  A quick look suggests that
  the problem is that transfer optimization is not disallowing cases where
  the foreign key constraints do not match on source and destination.

  <hr><i>drh added on 2011-04-24 23:02:35:</i><br>
  Confirmed: the problem was in the transfer optimization.  

  This problem has existed in all prior versions of SQLite going back version
  3.6.19 (2009-10-14) when foreign key constraints were first introduced.

  A work-around is to not use the "*" in the SELECT statement but instead
  to list the columns to be transfered:

  <blockquote><pre>
  INSERT INTO child SELECT s1 FROM source;
  </pre></blockquote>

  The work-around avoids the problem since the transfer optimization
  only works on INSERT INTO ... SELECT * ... statements.


tkt_uuid: 2d1a5c67dfc2363e44f29d9bbd57f7331851390a
type: Code_Defect
status: Fixed
subsystem: Pager
priority: Immediate
severity: Critical
foundin: 3.7.6
resolution: Fixed
title: Data loss on COMMIT in WAL mode
comment:
  The following SQL demonstrates the problem:

  <blockquote><verbatim>
  PRAGMA cache_size=4;
  PRAGMA journal_mode=WAL;
  CREATE TABLE t1(a,b);
  CREATE INDEX t1b ON t1(b);
  CREATE TABLE t2(x,y UNIQUE);
  INSERT INTO t2 VALUES(3,4);
  BEGIN;
  INSERT INTO t1(a,b) VALUES(1,2);
  SELECT 'A', * FROM t2 WHERE y=4;
  SELECT 'B', * FROM t1;
  COMMIT;
  SELECT 'C', * FROM t1;
  </verbatim></blockquote>

  The row inserted into t1 is present at B, but at C after the commit
  the t1 table is empty.  Changing the cache_size from 4 to 5 causes
  the problem to go away.

  <hr><i>drh added on 2011-05-19 01:28:33:</i><br>
  This problem has existed in all previous versions of SQLite that support WAL.

  The problem occurs when a multi-statement write transaction ends with one
  or more SELECT statements that cause a cache spill due to memory pressure.
  If there are no subsequent write operations, then when the transaction goes
  to commit, there might not be any dirty pages left to write out to the WAL
  with the commit-flag set.  And so the commit-flag does not get set and the
  transaction either merges into the next transaction, or if there is not
  next transaction, the transaction will silently roll back.

  We believe that the cache size must be set very small (less than 10 and
  probably less than 5) for this to happen.

  The fix is to always write page 1 to the WAL when there is a commit of
  a non-empty transaction, even if page 1 was not changed or was already
  written to WAL.


tkt_uuid: bd484a090c8077b8deebc20669f5613c4b19bcb2
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.7.6
resolution: Fixed
title: Date/time functions do not detect failure of localtime_r()
comment:
  The localtime_r() (or localtime_s() or localtime()) function can fail but
  that failure is not detected by SQLite.  An undefined answer can result.
  In builds that use localtime(), a NULL pointer is dereferenced.

  <hr><i>dan added on 2011-06-21 12:59:31:</i><br>

  Introduced by [8482b8c44766e7f8]. First appeared in version 2.8.9.

  <hr><i>rogerb added on 2011-09-19 20:43:56:</i><br>
  Note related ticket 0b803bff85 where getting the current time from a VFS results in errors being ignored.


tkt_uuid: 91e2e8ba6ff2e2b2abd93c60f3b87f093639ad7d
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.6.3
resolution: Fixed
title: Automatic indices cause undesirable type conversions
comment:
  Consider the following SQL code:

  <nowiki><blockquote><verbatim>
  CREATE TABLE customer (id INT PRIMARY KEY);
  CREATE TABLE apr (id INT PRIMARY KEY, apr REAL);

  CREATE VIEW v1 AS
  SELECT c.id, i.apr
  FROM customer c
  LEFT JOIN apr i ON i.id=c.id;

  CREATE VIEW v2 AS
  SELECT c.id, v1.apr
  FROM customer c
  LEFT JOIN v1 ON v1.id=c.id;

  INSERT INTO customer (id) VALUES (1);
  INSERT INTO apr (id, apr) VALUES (1, 12);
  INSERT INTO customer (id) VALUES (2);
  INSERT INTO apr (id, apr) VALUES (2, 12.01);

  --works as expected
  SELECT id, (apr / 100), typeof(apr) apr_type  FROM v1;
  --wrong type value for "apr"
  SELECT id, (apr / 100), typeof(apr) apr_type  FROM v2;
  </verbatim></blockquote></nowiki>

  The two queries at the end should give identical results, however
  the second query returns apr as an integer when it can be expressed
  as an integer, which is incorrecct.

  The problem was originally reported on the mailing list by Mark Brand.

  The problem was introduced by automatic indices with
  check-in [2364313142668b].  The issue is that REAL values are
  automatically convert into INT if they can be prior to storage
  in tables an indices.  This saves storage space since INT is
  typically smaller than REAL.  (A integer 0 takes up 1 bytes versus
  9 bytes for 0.0.)  The conversion back to REAL happens automatically
  for real tables.  But if the table is  on a manifestation of a view, 
  then the conversion back to REAL is omitted, resulting in an
  incorrect type on the result.

  The problem can be worked around by disabling automatic indices,
  which will prevent any indices on manifestations of views.


tkt_uuid: 25ee81271091ec27a8c5e00eeffc02949a712cb7
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.7
resolution: Fixed
title: PRAGMA case_sensitive_like=1 fails
comment:
  The following test program reports that the PRAGMA fails:

  <blockquote><verbatim>
  #include <stdio.h>
  #include "sqlite3.h"

  int main(void){
    sqlite3 *db3;
    char *errmsg;
    int err;

    sqlite3_open("test.db", &db3);
    err = sqlite3_exec(db3, "PRAGMA case_sensitive_like=1;", 0, 0, &errmsg);
    if (err != SQLITE_OK){
      printf("Error %d: %s\n", err, errmsg);
      sqlite3_free(errmsg);
    }
    return 0;
  }
  </verbatim></blockquote>

  This problem was reported on the mailing list by Greg Stein.  Investigation
  shows that the PRAGMA actually works; it simply returns an SQLITE_SCHEMA
  error code instead of SQLITE_OK.  The problem was introduced by
  check-in [957b2ab67c61] by the removal of "p->expired = 0;" from 
  the sqlite3VdbeAddOp3().  The case_sensitive_like function causes
  a new LIKE function to be registered, which sets "p->expired = 1",
  leading to the error.  It is unclear (yet) why this problem was not
  picked up by tests.


tkt_uuid: d6ddba6706353915ceedc56b4e3e72ecb4d77ba4
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.7
resolution: Fixed
title: Parser accepts invalid CREATE TRIGGER syntax
comment:
  The parser accepts invalid CREATE TRIGGER syntax without complaint, which
  can result in content of the SQLITE_MASTER table that fails at a later time.

  The name of the table that the trigger watches should not be allowed to
  include a database name.  For example:

     CREATE TRIGGER ex1 AFTER INSERT ON main.table1 BEGIN SELECT 0; END;

  The "main" on "main.table1" should not be allowed.  The syntax diagrams
  clearly show that you should only have "table1" at that point.  

  The example above will work, but if the database name is for an attached
  database, then when the database is opened by any other attachment name
  the open will fail.

  The problem was reported on the mailing list by Filip Navara.
  The problem goes back to before SQLite 3.1.3 (February 2005).


tkt_uuid: 54844eea3f20252442263028908948a1f1eb4d3f
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: 3.7.7
resolution: Fixed
title: Incorrect caching of sub-query results in the FROM clause of a scalar sub-query.
comment:
  If a sub-query that is in the FROM clause of a correlated scalar sub-query contains a reference to the outer query (the parent of the scalar sub-query), then the results of that FROM clause sub-query will change each time the scalar query is executed. It is therefore not possible to create an automatic index on the results of such a sub-query. 

  This bug is that currently, SQLite sometimes does create such automatic indexes, effectively caching the results of the sub-query the first time it is run.

  <verbatim>
      CREATE TABLE t4(a, b, c);
      INSERT INTO t4 VALUES('a', 1, 'one');
      INSERT INTO t4 VALUES('a', 2, 'two');
      INSERT INTO t4 VALUES('b', 1, 'three');
      INSERT INTO t4 VALUES('b', 2, 'four');
      SELECT (
        SELECT c FROM (
          SELECT * FROM t4 WHERE a=out.a ORDER BY b LIMIT 10 OFFSET 1
        ) WHERE b=out.b
      ) FROM t4 AS out;
  </verbatim>

  i.e. in the example above the results of "SELECT * FROM t4 WHERE a=out.a ORDER BY b LIMIT 10 OFFSET 1" are being incorrectly cached the first time the subquery is executed, leading to an incorrect result.

  <hr><i>dan added on 2011-07-08 16:00:37:</i><br>

  This bug has been present since automatic indexes were implemented for version 3.7.0. Commit [bf7b8d8636].


  <hr><i>drh added on 2011-07-08 17:05:34:</i><br>
  This problem was found during internal testing. It has never been observed
  in the wild.


tkt_uuid: 534aa198e11f8c171b0344c1852ebd23d48eba0f
type: Build_Problem
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.4
private_contact: 8aad979c37a6a20afcd1bac9b0570bf6d02ba0e7
resolution: Not_A_Bug
title: Build error: missing manifest.uuid file in 3.7.4 downloadable source code
comment:
  To Whom it May Concern,

  I am trying to build sqlite 3.7.4 from the downloadable zip/tar files found in:
  Check-in [a586a4deeb]
  The build fails:
  make: *** No rule to make target `../manifest.uuid', needed by `sqlite3.h'.  Stop.
  Because the manifest.uuid file is not included in the source tree.
  The manifest file is available, but I don't know if it is in the proper format.

  Thank you,
  Alan

  <hr><i>drh added on 2011-07-14 11:38:10:</i><br>
  This was due to a configuration error on the www2.sqlite.org and www3.sqlite.org
  servers.  (The www.sqlite.org server was find.)  The configuration issue
  has now been resolved.


tkt_uuid: 6c14288a473ceffbe7c07b1136130746449cab03
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.7
resolution: Fixed
title: Virtual tables do not handle IS NOT NULL correctly
comment:
  When SQLITE_ENABLE_STAT2 is enabled, the IS NOT NULL constraint is not handled
  correctly by virtual tables.  This problem was introduced by the IS NOT NULL
  optimization of check-in [499edcbc8ab70fcf35431d4].  The problem first appeared
  in release 3.7.6.


tkt_uuid: d63523637517386191d634eaa71de2c7ecf1e96e
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.7.7
resolution: Fixed
title: Assertion fault on a join with a repeated column
comment:
  When SQLite is compiled with -DSQLITE_DEBUG, an assert() fault occurs when
  the following SQL is run:

  <verbatim>
     CREATE TABLE t1(id1 INTEGER PRIMARY KEY);
     INSERT INTO t1 VALUES(9999);
     CREATE TABLE t2(id2 INTEGER PRIMARY KEY);
     INSERT INTO t2 VALUES(12345);
     INSERT INTO t2 VALUES(54321);
     SELECT count(*) FROM t1, t2 GROUP BY id1, id1;
  </verbatim>

  The failing assert() is checking to ensure that when a "shallow copy" of
  a value is used, that the original hard copy of the object that the
  shallow copy is pointing to is unchanged from when the shallow copy was
  first made.  In this case, both the shallow copy and the original are moving
  together, though, so the assertion fault does not indicate a real malfunction.
  The work-around is to not compile with SQLITE_DEBUG.  A
  proper fix is forthcoming.


tkt_uuid: e59bdf6116036a2542f518f111612df09154661e
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.7
resolution: Fixed
title: Possible database corruption following EINTR on unix
comment:
  It is reported to us that on some unix systems, if an EINTR error occurs on
  a write() system call, it might be that some of the write() completed and
  thus advanced the file pointer.  Thus if the write() is simply retried without
  first doing an lseek() to reposition the file pointer, the buffer will be
  written into the wrong place.

  This is a theoretical problem only.  We have no idea if this scenario can
  actually occur on any real unix platforms.  We have never actually seen this
  problem occur in a working system.

  The solution to the problem (if it is a problem at all) is to simply include
  the lseek() call in the do-loop that retries the write() following an
  EINTR error.


tkt_uuid: b1d3a2e531478b4e069df3d2e847b8d4346ce4ce
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Minor
foundin: 3.7.7.1
resolution: Fixed
title: Erroneous FK constraint failure when dropping table.
comment:
  The following script causes a constraint failure.

  <verbatim>
     CREATE TABLE pp(x PRIMARY KEY);
     CREATE TABLE cc(y REFERENCES pp DEFERRABLE INITIALLY DEFERRED);
     INSERT INTO pp VALUES('a value');
     INSERT INTO cc VALUES('a value');

     BEGIN;
       DROP TABLE pp;
       DROP TABLE cc;
     COMMIT;
  </verbatim>

  The problem is that SQLite sees that the "DROP TABLE pp" creates a constraint violation (since there is now an orphan row in "cc"). However it does not see that "DROP TABLE cc" corrects the violation (by deleting the orphan). This is an oversight in the implementation of foreign-keys.

  The bug has been present ever since FK support was introduced, in version 3.6.19.

  <hr><i>dan added on 2011-08-22 09:55:58:</i><br>
  Fixed here: [3f3acee465]


tkt_uuid: af95b8c609b5571a4e6397ea64e80b9fe4c4a545
type: Code_Defect
status: Fixed
subsystem: Utilities
priority: Immediate
severity: Severe
foundin: 3.7.7.1
resolution: Fixed
title: Vacuuming a WAL database resets the file header read and write versions to 1.
comment:
  After a WAL database is VACUUMed, the file header read and write version fields at byte offsets 18 and 19 of the database file are left set to 1. They should be set to 2 for a WAL database.

  The effect of this is that following a VACUUM, all existing connections keep using the database in WAL mode. And all new connections made while old connections still exist also use the database in WAL mode. But if the number of connections drops to zero (with a clean shutdown - meaning the *-wal file is deleted), then the next client to open the database file opens in in WAL mode.

  This is not at all dangerous from a data corruption point of view. It is just annoying - in that at some point following a VACUUM the system magically switches back to rollback mode.


tkt_uuid: 56be9768592940273e30278d29639f265f08df02
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Severe
foundin: 3.7.7.1
resolution: Fixed
title: Buffer overrun in FTS prefix queries
comment:
  A buffer overrun may occur in FTS when executing a prefix query if either (a) the FTS table uses order=DESC, or (b) if the FTS table contains negative rowid values.

  <hr><i>dan added on 2011-09-14 11:13:49:</i><br>
  This bug was introduced by [551dbe30eab585]. The first release to contain the problem was 3.6.21.




  <hr><i>dan added on 2011-09-14 11:16:23:</i><br>
  Fixed by [07788c0f7f].


tkt_uuid: 002caede898aee4dc531ae9995281c2b8b03a1c6
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.7.1
resolution: Fixed
title: LEFT JOIN with OR terms in WHERE clause causes assertion fault
comment:
  The following SQL code causes an assertion fault in the VDBE:

  <blockquote><pre>
  CREATE TABLE t1(a,b,c,d);
  CREATE TABLE t2(e,f);
  INSERT INTO t1 VALUES(1,2,3,4);
  INSERT INTO t2 VALUES(10,-8);
  CREATE INDEX t1a ON t1(a);
  CREATE UNIQUE INDEX t1b ON t1(b);
  CREATE TABLE t3(g);
  INSERT INTO t3 VALUES(4);
  CREATE TABLE t4(h);
  INSERT INTO t4 VALUES(5);

  SELECT * FROM t3 LEFT JOIN t1 ON d=g LEFT JOIN t4 ON c=h
   WHERE (a=1 AND h=3)
       OR b IN (
             SELECT x+1 FROM (SELECT e+f AS x, e FROM t2 ORDER BY 1 LIMIT 2)
             GROUP BY e
          );
  </pre></blockquote>

  It appears that both LEFT JOINs, the OR operator using indices in the
  WHERE clause, and the IN operator with unflattenable 
  nested subqueries and located on the right-hand
  side of the OR are all necessary conditions for this problem to occur.
  In other words, the problem is very obscure.

  The problem appears to have been introduced by check-in
  [2c2de25266] which was a bug-fix for a similar problem described by
  ticket [31338dca7e3d].  This problem might be considered a particular
  case of the prior ticket [31338dca7e3d] that was never fixed.

  The problem first appeared in release 3.6.22.


tkt_uuid: 1a1308d2538d7ebb467fb112ba0abc1ba9f91507
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Severe
resolution: Fixed
title: Nested correlated subquery fails when indices used
comment:
  Consider the following SQL:

  <blockquote><pre>
  CREATE TABLE t1(a,b);
  INSERT INTO t1 VALUES(1,2);
  INSERT INTO t1 VALUES(2,4);
  INSERT INTO t1 VALUES(99,101);
  INSERT INTO t1 VALUES(98,97);
  CREATE TABLE t2(e,f);
  INSERT INTO t2 VALUES(10,-8);
  CREATE INDEX t1b ON t1(b);

  SELECT 'one:', * FROM t1 
   WHERE b IN (SELECT x FROM (SELECT e+f+a AS x, e FROM t2 ORDER BY 1 LIMIT 2)
               GROUP BY e);
  SELECT 'two:', * FROM t1 
   WHERE +b IN (SELECT x FROM (SELECT e+f+a AS x, e FROM t2 ORDER BY 1 LIMIT 2)
               GROUP BY e);
  </pre></blockquote>

  The two queries should give identical results.  The only difference is the
  unary "+" operator on the "b" of the LHS of the IN operator in the second
  query.  That unary + disables the use of indices for trying to evaluate
  the IN operator.  

  But they give different results.  We believe the second one (with the unary +)
  is giving the correct result.

  This problem was introduced by check-in [daf730d1defa7] which was a refactoring
  of the name resolution logic.  Prior to that check-in, these queries gave an
  error "no such column: a".  So the problem is probably older than the check-in
  cited, but was not accessible until that check-in.

  The problem was discovered while writing additional test cases for
  ticket [002caede898aee] though this problem appears to be unrelated to 
  that other ticket.


tkt_uuid: 57c971fc74524aa46e0b367dbed9a9a77aed4ebc
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Minor
foundin: 3.7.8
resolution: Fixed
title: Some user defined string functions do not operate correctly on strings with embedded nuls
comment:
  SQLite supports embedded NUL in strings, both via the C API and SQL.  For example this will create one:

    INSERT INTO table VALUES('abc' || x'00' || 'def');

  Many of the user defined string functions such as quote, upper/lower, length, replace, (r)trim stop their operations at the first NUL rather than at the end of the string.

  Somewhat related you can also create UTF16 sequences with odd numbers of bytes using the insert sequence above because || operates on bytes not characters.

  This topic was discussed on the mailing list:

    http://thread.gmane.org/gmane.comp.db.sqlite.general/68307/focus=68400

  <hr><i>drh added on 2011-10-13 18:01:28:</i><br>
  Updated documentation to try to make it clearer that the result of
  expressions involving strings with embedded NUL characters is undefined.
  Also added more detail to the documentation on sqlite3_prepare(), 
  sqlite3_bind_text(), and sqlite3_result_text() to try to clear up confusion
  about the use of NUL terminators.


tkt_uuid: d1d84037b90a44966c67827086526b3bdff22cba
type: Feature_Request
status: Fixed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.7.8
resolution: Fixed
title: import doesn't work on attached databases due to unnecessary quoting
comment:
  Trying to import to an attached database fails.  This is because the code does this:

  <verbatim>
    sqlite3_mprintf("SELECT * FROM '%q'", zTable);
    sqlite3_snprintf(... "INSERT INTO '%q' VALUES(?", zTable);
  </verbatim>

  If you put single quotes around a table name then database names within are not parsed.

  <verbatim>
    sqlite> attach '/tmp/foo' as foo;
    sqlite> create table foo.bar(x);
    sqlite> select * from foo.bar;
    sqlite> select * from 'foo.bar';
    SQLError: no such table: foo.bar
  </verbatim>

  The code should not be using single quotes or SQLite should be changed to parse database names when single quotes are used.


tkt_uuid: c48d99d690180b6933052732e3754cf4f2ff611b
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: 3.7.8
resolution: Fixed
title: Crash in "INSERT INTO ... SELECT" when "PRAGMA count_changes=1" is configured
comment:
  If "PRAGMA count_changes=1" is configured, and an "INSERT INTO ... SELECT" statement that uses the xfer optimization is executed, an assertion fails. Or, if the library is built without SQLITE_DEBUG, an invalid location is read and written. Which sometimes causes a segfault.

  <verbatim>
  SQLite version 3.7.9 2011-09-26 19:32:47
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a, b);
  sqlite> INSERT INTO t1 VALUES(1, 2);
  sqlite> CREATE TABLE t2(c, d);
  sqlite> INSERT INTO t2 SELECT * FROM t1;
  sqlite> PRAGMA count_changes=1;
  sqlite> INSERT INTO t2 SELECT * FROM t1;
  sqlite3: /home/dan/work/sqlite/trunk/src/vdbe.c:1103: sqlite3VdbeExec: Assertion `pOp->p1>0' failed.
  Aborted (core dumped)
  </verbatim>

  <hr><i>dan added on 2011-09-30 12:02:28:</i><br>
  Fixed ([9ddfe1e413]) by disabling xfer optimization if "PRAGMA count_changes=1" is configured.


  <hr><i>dan added on 2011-09-30 12:19:04:</i><br>

  This bug has been present in one form or another since 3.3.14. In earlier versions there was no assert() or bad memory reference, but the INSERT statement was not returning the number of rows modified either.


tkt_uuid: fa7bf5ec94801e7e2030e41eefe5d9dd96eaacfd
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.8
resolution: Fixed
title: String literal case confusion in aggregate query
comment:
  Consider the following SQL:

  <blockquote><pre>
  CREATE TABLE t1(x);
  INSERT INTO t1 VALUES ('a');
  INSERT INTO t1 VALUES ('A');
  INSERT INTO t1 VALUES ('A');
  SELECT count(CASE WHEN x='a' THEN 1 END),
         count(CASE WHEN x='A' THEN 1 END)
    FROM t1;
  </pre></blockquote>

  The SELECT at the end returns "1|1" instead of "1|2" as it should.

  <hr><i>drh added on 2011-10-13 15:38:08:</i><br>
  This problem has existed in the code since at least version 3.1.3
  (2005-02-19) and probably well before that.


tkt_uuid: 9fd058691b566188071be12f6983ec40d509d188
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Critical
foundin: 3.7.7 through 3.7.8
resolution: Fixed
title: Problem with updating an FTS table that contains a single row
comment:
  If the only row in an FTS table is updated and the rowid/docid value is not changed, then the FTS index may become corrupt.

  For example:

  <verbatim>
  $ ./sqlite3 x
  SQLite version 3.7.9 2011-10-13 16:36:29
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE VIRTUAL TABLE t USING fts3;      
  sqlite> INSERT INTO t VALUES('xxx');
  sqlite> .quit
  </verbatim>
  Then:
  <verbatim>
  $ ./sqlite3 x
  SQLite version 3.7.9 2011-10-13 16:36:29
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> UPDATE t SET content = 'yyy';
  sqlite> SELECT * FROM t WHERE t MATCH 'yyy';
  Error: database disk image is malformed
  </verbatim>

  <hr><i>dan added on 2011-10-13 17:33:07:</i><br>
  Introduced here [8f9666af]. Bug first appeared in release 3.7.7.

  <hr><i>dan added on 2011-10-13 17:34:50:</i><br>
  Fixed here: [7e24645be2] 


tkt_uuid: 8290242b2a9a816830a0e34e6429c631ca80b558
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.8
resolution: Fixed
title: ALTER TABLE RENAME fails if the database encoding is not UTF8
comment:
  ALTER TABLE RENAME passes the new name of a virtual table to the xRename
  method of the virtual table in the database encoding instead of in UTF8.
  This causes most xRename implementations to malfunction.  For example:

  <blockquote><pre>
  PRAGMA encoding=UTF16le;
  CREATE VIRTUAL TABLE rt USING rtree(i,x1,x2,y1,y2);
  CREATE VIRTUAL TABLE fts USING fts4(one,two);
  .tables
  ALTER TABLE rt RENAME TO x;
  ALTER TABLE fts RENAME TO y;
  .tables
  </pre></blockquote>


tkt_uuid: 48f299634acb013ba05730b8d86bb029fc4372f2
type: Code_Defect
status: Closed
subsystem: Virtual_Machine
priority: Immediate
severity: Important
foundin: 3.7.8
resolution: Fixed
title: Crash in SAVEPOINT within a transaction that writes to then drops an FTS table
comment:
  The following script causes a crash:

  <verbatim>
  CREATE VIRTUAL TABLE f1 USING fts3;
  BEGIN; 
    INSERT INTO f1 VALUES('a b c');
    DROP TABLE f1;
    SAVEPOINT one;
  </verbatim>

  This is not an FTS specific problem. It can occur using any virtual table implementation that sets sqlite3_module.iVersion>=2 (i.e. one that requires callbacks for SAVEPOINT, ROLLBACK TO and RELEASE commands).

  To reproduce this, the user must open a transaction, write to the virtual table and then DROP it. Thereafter, any SAVEPOINT, ROLLBACK TO or RELEASE command within the transaction that is not a no-op will cause a crash.

  <hr><i>dan added on 2011-10-29 11:46:05:</i><br>
  Fixed here: [3565fcf898].



  <hr><i>dan added on 2011-10-29 11:54:59:</i><br>
  First appeared in version 3.7.7.


tkt_uuid: 676bc02b87176125635cb174d110b431581912bb
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.9
resolution: Fixed
title: Incorrect xfer optimization with INTERGER PRIMARY KEY ON CONFLICT REPLACE
comment:
  An INTEGER PRIMARY KEY ON CONFLICT REPLACE on the destination table
  causes an INSERT from one table into another to fail rather than REPLACE
  when the xfer optimization is used.  The following code illustrates:

  <blockquote><pre>
  CREATE TABLE t1(a INTEGER PRIMARY KEY ON CONFLICT REPLACE, b);
  CREATE TABLE t2(x INTEGER PRIMARY KEY ON CONFLICT REPLACE, y);
  INSERT INTO t1 VALUES(1,2);
  INSERT INTO t2 VALUES(1,3);
  INSERT INTO t1 SELECT * FROM t2;
  SELECT * FROM t1;
  </pre></blockquote>

  The result of the above should be 1|3 but it instead generates an error.


tkt_uuid: 3a77c9714e63330eee5001c88096c52da06231d9
type: Code_Defect
status: Fixed
subsystem: Parser
priority: Immediate
severity: Critical
foundin: 3.7.9
resolution: Fixed
title: Malfunction involving temporary registers and sub-queries implemented by vdbe sub-routines.
comment:
  The following SQL returns no rows. Where it should return a
  single row.

  <verbatim>
    CREATE TABLE t1(t1_id INTEGER PRIMARY KEY, t1_title TEXT);
    CREATE TABLE t2(t2_id INTEGER PRIMARY KEY, t2_title TEXT);
    CREATE TABLE t3(t3_id INTEGER PRIMARY KEY, t3_title TEXT);

    INSERT INTO t1 (t1_id, t1_title) VALUES (888, 'FACTORING');
    INSERT INTO t2 (t2_id, t2_title) VALUES (999, 'FACTORING');
    INSERT INTO t3 (t3_id, t3_title) VALUES (999, 'FACTORING');

    SELECT t1_title, t2_title
    FROM t1 LEFT JOIN t2
    WHERE
      t2_id = (SELECT t3_id FROM
       ( SELECT t3_id FROM t3 WHERE t3_title=t1_title LIMIT 500 )
    )
  </verbatim>

  The problem is that the sub-query:

    ( SELECT t3_id FROM t3 WHERE t3_title=t1_title LIMIT 500 )

  is implemented using a vdbe sub-routine (OP_Gosub/OP_Return) that
  is called from two places. When called the second time, it overwrites the contents of temporary registers that are being used by the caller to store values. Causing the malfunction.

  This manifestation of the problem was introduced by [2c2de25266], although it is possible that there are other obscure ways to trigger a similar bug in earlier versions.


tkt_uuid: 7bbfb7d4422ff413b065b410bf9f3dd00a4dd2a6
type: Code_Defect
status: Fixed
subsystem: Virtual_Machine
priority: Immediate
severity: Important
foundin: 3.7.9
resolution: Fixed
title: Problem with using stale auto-indexes created on materialized sub-selects within trigger programs
comment:
  Sometimes when a query features a non-correlated sub-select, SQLite executes the query and stores the results in an ephemeral table. If required, it may also generate an automatic-index based on the contents of the ephemeral table. If it
  does so, it generates the automatic-index lazily - when it is first required by the VDBE at runtime. Once the automatic index has been generated, a flag is set by the OP_Once opcode to indicate that this has taken place and the index does not need to be generated again.

  If all this happens within a trigger program, both the automatic index and the flag persist for the lifetime of the sqlite3_step() call, even if the trigger program is invoked more than once (either recursively or because the calling statement modifies or inserts more than one row). The ephemeral table on the other hand, is regenerated each time the trigger program is invoked.

  The bug is that even though the sub-select may be non-correlated from the point of view of the trigger program, it may depend on the values of new.* or old.* registers. And so it may change for each invocation of the trigger program. Since the index will not be updated in this case, the second and subsequent invocations of the trigger program can end up using an automatic-index filled with the wrong data. Obviously this can cause a malfunction.

  For example:

  <verbatim>
    CREATE TABLE t1(a, b);
    INSERT INTO t1 VALUES(1, 'one'); 
    INSERT INTO t1 VALUES(2, 'two');
    INSERT INTO t1 VALUES(3, 'three');

    CREATE TABLE t2(c, d);
    INSERT INTO t2 VALUES('one', 'I');
    INSERT INTO t2 VALUES('two', 'II');
    INSERT INTO t2 VALUES('three', 'III');

    CREATE TABLE t3(t3_a PRIMARY KEY, t3_d);
    CREATE TRIGGER t3t AFTER INSERT ON t3 WHEN new.t3_d IS NULL BEGIN
      UPDATE t3 SET t3_d = (
        SELECT d FROM
          (SELECT * FROM t1 WHERE (new.t3_a%2)=(rowid%2) LIMIT 10),
          (SELECT * FROM t2 WHERE (new.t3_a%2)=(rowid%2) LIMIT 10)
        WHERE a = new.t3_a AND b = c
      ) WHERE t3_a = new.t3_a;
    END;
  </verbatim>

  Then:

  <verbatim>
  sqlite> INSERT INTO t3(t3_a) VALUES(1);
  sqlite> INSERT INTO t3(t3_a) VALUES(2);
  sqlite> INSERT INTO t3(t3_a) VALUES(3);
  sqlite> SELECT * FROM t3;
  1|I
  2|II
  3|III
  sqlite> DELETE FROM t3;
  sqlite> INSERT INTO t3(t3_a) SELECT 1 UNION SELECT 2 UNION SELECT 3;
  sqlite> SELECT * FROM t3;
  1|I
  2|
  3|III
  </verbatim>

  When the three rows above are inserted using discrete INSERT statements, the
  automatic-index associated with the sub-select in the trigger program is
  generated each time a row is inserted. And things work. However, when all three
  rows are inserted using a single INSERT, the automatic-index is generated once
  when the first row is inserted (the row with t3_a==1) and reused by each
  invocation of the trigger program thereafter. Since the index generated for
  (t3_a==1) does not include an entry for the tuple (2, 'two', 'two', 'II'), the
  second invocation of the trigger program (with t3_a==2) malfunctions and
  writes a NULL into the table instead of the string 'II'.

  <b>Note:</b> For the example dataset, the trigger in the above is equivalent to the following. The inner sub-select is convoluted a bit in order to prevent SQLite from applying the flattening optimization, and so that its results depend on the value of "new.t3_a" when the trigger program is invoked.

  <verbatim>
    CREATE TRIGGER t3t AFTER INSERT ON t3 WHEN new.t3_d IS NULL BEGIN
      UPDATE t3 SET t3_d = (
        SELECT d FROM t1, t2 WHERE a = new.t3_a AND b = c
      ) WHERE t3_a = new.t3_a;
    END;
  </verbatim>


tkt_uuid: a1fa75cbdd02eebbbf426745dc53c7ed3b6cafbe
type: Code_Defect
status: Fixed
subsystem: Pager
priority: Immediate
severity: Important
foundin: 3.7.9
resolution: Fixed
title:
  Database corruption following unlucky application crash in WAL mode with "PRAGMA journal_size_limit"

comment:
  In WAL mode, if a journal-size limit is configured using "PRAGMA journal_size_limit", then the WAL file is truncated to the configured size as part of the first database write following each completed checkpoint. If the
  application crashes immediately after performing this truncate, the database can be left in a corrupt state.

  <hr><i>dan added on 2011-12-16 13:28:34:</i><br>
  Introduced here: [db7e500f69]

  First appeared in release 3.7.7.


tkt_uuid: ff5be73dee08608976bd32851fa07b60381609ca
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.9
resolution: Fixed
title: Database corruption following power-loss in WAL mode
comment:
  A carefully timed power loss on a filesystem that does out-of-order writes
  can cause database corruption when operating in WAL mode.  The setup for the
  corruption scenario is as follows:

    1.  Write many small transactions
    2.  Checkpoint
    3.  Begin writing a large transaction
    4.  Power-loss with out-of-order write prior to the transaction fsync

  If transaction fsyncs are disabled because PRAGMA synchronous is set to NORMAL,
  then the power-loss can occur at any time after SQLite begins overwriting
  the WAL file again from the beginning.

  If out-of-order writing causes disk sectors after the first transaction
  of the previous cycle of the WAL to be overwritten and earlier sectors to
  remain unchanged, then after reboot the first
  process to connect will find a valid WAL file containing one or more
  transactions from the previous cycle of the WAL
  and it will replay those transactions.  But subsequent transactions in that
  same the WAL cycle
  will be lost due to the overwrite and will not be replayed.  The later
  transactions will already exist in the database file, however, meaning that
  transactions will occur out-of-order.  This can easily result in database
  corruption.

  This problem has existed in all prior versions of SQLite that support WAL
  mode.  This problem was detected during internal stress testing and has
  not been observed in the wild.  We believe that the behavior of real-world
  filesystems is such that this fault, while theoretically possible, is 
  unlikely to occur in practice.

  <hr>
  Further notes:

  In most filesystems, space to hold a file is allocated in sequential
  sectors on disk.  Filesystems do this because it is usually faster to
  read sectors sequentially rather than randomly.  There can be 
  discontinuities in the sector sequence for larger files, but the beginning
  of the file in particular is usually arranged contiguously on disk.

  Similarly, when writing to disk, filesystems tend to order sector writes
  sequentially.  (See, for example, the
  [http://en.wikipedia.org/wiki/Elevator_algorithm | elevator algorithm].)

  Disk controllers might also reorder writes, but they too seem likely
  to perform writes in sector order when possible.

  When you consider the three factors above, it seems that the common case
  behavior when overwriting a file sequentially will be that
  content will reach disk oxide sequentially as well.  And when that 
  happens, the failure scenario described by this ticket will not occur.

  Combine this with the fact that database damage also requires a well-timed 
  power loss or hard reset (which we also hope is a rare event) and one
  begins to understand that the failure scenario described by this
  ticket is a very unlikely thing to happen.

  The problem is still worth fixing, to be sure.  But in consideration of
  the above factors, we have downgraded the severity from Critical to Severe.
  We will push forward the next release of SQLite in order to get a fix 
  into circulation sooner rather than later.  But we do not feel
  compelled to rush out a patch release nor take heroic measures to make
  the 3.7.10 release during the Christmas holiday.


tkt_uuid: ac0ff496b7e21147e9985332256389074bc7722d
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.9
resolution: Fixed
title: Database corruption due to integer overflow with SQLITE_DIRECT_OVERFLOW_READ
comment:
  When the (non-standard) compile-time option SQLITE_DIRECT_OVERFLOW_READ
  is used, a 32-bit integer overflow at [/artifact/80ea65224512?ln=3982]
  will likely cause the database to be reported as corrupt.  The database
  is not really harmed by this, but attempts to read the database will be
  unsuccessful.

  This bug has existed since the SQLITE_DIRECT_OVERFLOW_READ option was added
  on [2011-10-08] by check-in [2ab14a8467278571].


tkt_uuid: edb497982cb45206a6561f82d47efc8cbe438bc3
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.10
resolution: Fixed
title: FTS n-character prefix query returns incorrect results when using an n+1 character prefix index.
comment:
  Sometimes, after rows have been deleted from an FTS table, querying
  for an N character prefix when there is only an N+1 character prefix
  available results in rowids for deleted rows being returned.

  In the following, FTS reports that there are 158 rows matching a prefix
  query on a table in which only 5 rows exist. Attempting to retrieve the
  row content causes an SQLITE_CORRUPT error (since the content entries have
  already been deleted).

  <verbatim>
  SQLite version 3.7.10 2012-01-14 03:34:24
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE VIRTUAL TABLE t1 USING fts4(x, prefix=2);
  sqlite> BEGIN;
  sqlite> INSERT INTO t1 VALUES('T TX T TX T TX T TX T TX');
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> INSERT INTO t1 SELECT * FROM t1;
  sqlite> COMMIT;
  sqlite> SELECT count(*) FROM t1;
  2048
  sqlite> INSERT INTO t1 SELECT * FROM t1 LIMIT 10;
  sqlite> INSERT INTO t1 SELECT * FROM t1 LIMIT 10;
  sqlite> INSERT INTO t1 SELECT * FROM t1 LIMIT 10;
  sqlite> DELETE FROM t1 WHERE docid > 5;
  sqlite> SELECT count(*) FROM t1;
  5
  sqlite> SELECT count(*) FROM t1 WHERE t1 MATCH 'T*';
  158
  sqlite> SELECT * FROM t1 WHERE t1 MATCH 'T*';
  T TX T TX T TX T TX T TX
  T TX T TX T TX T TX T TX
  T TX T TX T TX T TX T TX
  T TX T TX T TX T TX T TX
  T TX T TX T TX T TX T TX
  Error: database disk image is malformed
  </verbatim>


tkt_uuid: c04a8b8a4f0c154cfdbd8a4a3086d04c7ed04027
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.10
resolution: Fixed
title: Trailing comments not stripped from views, invalid dumps
comment:
  Reported on the mailing list.  When making a dump the SQL text from sqlite_master is taken as is and a semi-colon appended.  For tables, triggers and indexes SQLite strips trailing comments before putting their text into sqlite_master.  However this is not done for views.  Consequently the view text can end up like:

  <pre>create view ......   -- a comment</pre>

  When this is dumped the trailing comment swallows any semicolon appended.

  Mailing list thread:  <a href="http://thread.gmane.org/gmane.comp.db.sqlite.general/71502">http://thread.gmane.org/gmane.comp.db.sqlite.general/71502</a>


tkt_uuid: 232637c465fd13df525a3116c1ea77c03e660d42
type: Code_Defect
status: Fixed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.7.10
resolution: Fixed
title: Dump code fails with 1002 columns
comment:
  Reported on the mailing and confirmed:  <a href="http://thread.gmane.org/gmane.comp.db.sqlite.general/71527">http://thread.gmane.org/gmane.comp.db.sqlite.general/71527</a>

  When you have more than ~1,000 columns, the SQL text constructed in dump_callback() that generates the INSERT statement results in SQLITE_ERROR:  Expression tree is too large (maximum depth 1000).

  This could be addressed by generating the column values another way such as retrieving them directly rather than via formatting SQL statements as queries.


tkt_uuid: 5f41597f7c9c1750a57e515afb7748499050a9e8
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.10
resolution: Fixed
title: sqlite3_uri_boolean defaults to true, documented as false, is a mess
comment:
  sqlite3_uri_boolean is documented as "<b>and is false otherwise</b>".

  The implementation calls sqlite3GetBoolean which then calls getSafetyLevel taking the lowest bit returned.  getSafetyLevel is documented like this:

  <pre>
  ** Interpret the given string as a safety level.  Return 0 for OFF,
  ** 1 for ON or NORMAL and 2 for FULL.  Return 1 for an empty or 
  ** unrecognized string argument.
  </pre>

  The default is the opposite of what sqlite3_uri_boolean expects.

  Additionally getSafetyLevel looks at the first character of the string and if it is a digit then returns sqlite3Atoi of the string which also contradicts the doc.

  <table border=1>
  <tr><th>value</th><th>sqlite3_uri_boolean doc</th><th>what is actually returned</th></tr>
  <tr><td>3</td><td>0</td><td>1</td></tr>
  <tr><td>(zero length)</td><td>0</td><td>1</td></tr>
  <tr><td>1fdsfds</td><td>0</td><td>1</td></tr>
  </table>

  I'd strongly prefer changing the behaviour to match the doc.


tkt_uuid: ae6794effd4049a6bf658efdc366dc08f03f8918
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.10
resolution: Fixed
title: ALTER TABLE RENAME fails for triggers that use the table name with a different case
comment:
  If the name of a table in a TRIGGER definition is in a different case than
  the name of the table in its CREATE TABLE statement, then an ALTER TABLE RENAME
  will not modify the name in the trigger.  For example, the following will fail:

  <blockquote><pre>
  CREATE TABLE t1(a,b,c);
  CREATE TABLE log(x,y,z);
  CREATE TRIGGER r1 AFTER INSERT ON T1 BEGIN
    INSERT INTO log VALUES(new.a,new.b,new.c);
  END;
  INSERT INTO t1 VALUES(1,2,3);
  ALTER TABLE t1 RENAME TO t2;
  INSERT INTO t2 VALUES(4,5,6);
  SELECT * FROM log;
  </pre></blockquote>

  Because the "T1" in the CREATE TRIGGER statement uses an upper-case "T"
  while the CREATE TABLE statement uses a lower-case "t" for the name of the
  "t1" table, the CREATE TRIGGER statement is not correctly modified by the
  "ALTER TABLE t1 RENAME TO t2" and the trigger becomes disassociated with
  the table.

  This problem was reported on the mailing list.


tkt_uuid: 72b01a982a84f64d4284f63907224c548470d457
type: Feature_Request
status: Closed
subsystem: Unknown
priority: Immediate
severity: Important
resolution: Rejected
title: Per page/region checksums
comment:
  Please add per page (or collection of pages) checksums.  This is to detect the case when what SQLite wrote to the database is different than what is later returned from the database.

  It is increasingly the case that files get larger, I/O systems more complex, features and voltages smaller at the physical level, more diversity in storage and OS (desktops, servers, mobiles, storage hierarchies).  A bug or stray electron could cause an unwanted change which won't be caught by integrity check pragma unless it happens in sufficiently important metadata.  Changes in data or valid but wrong metadata (eg an index entry pointing to wrong row id) will be silent.

  The expectation is that checksums would be checked on each page read plus the whole database on pragma integrity check.

  It is understood that it will slow down things somewhat and may impose a version bump (ie older SQLites that do not understand checksums should not be able to write to a db that uses them).


tkt_uuid: 3557ad65a076ca6bb71d57d6fba0e7e1bad7e425
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
foundin: 3.7.10
resolution: Fixed
title: Incorrect DISTINCT on an indexed query with IN
comment:
  Reported on the mailing list.  The following SQL gives an incorrect result:

  <blockquote><pre>
  CREATE TABLE t1(a,b);
  INSERT INTO t1 VALUES(1,1);
  INSERT INTO t1 VALUES(2,1);
  INSERT INTO t1 VALUES(3,1);
  INSERT INTO t1 VALUES(2,2);
  INSERT INTO t1 VALUES(3,2);
  INSERT INTO t1 VALUES(4,2);
  CREATE INDEX t1ab ON t1(a,b);
  SELECT DISTINCT b from t1 WHERE a IN (1,2,3);
  </pre></blockquote>

  <hr><i>drh added on 2012-03-03 01:35:55:</i><br>
  Introduced by check-in [45e581bff7a75d] on 2011-07-02 for release 3.7.8.


tkt_uuid: f48b7da35c522d9b9564a23492460faff9d55198
type: Documentation
status: Closed
subsystem: Documentation
priority: Immediate
severity: Important
resolution: Overcome_By_Events
title: Loadable extensions documentation bitrot
comment:
  The documentation for loadable extensions has bit rotted.  Look at these two searches:

  https://www.google.com/search?q=sqlite+loadable+extensions

  http://www.sqlite.org/search?q=loadable+extensions


  None of the API docs under /c3ref/ link to a page about them.  This page
  mentions an old (and unlinked) cvstrac ticket:

    http://www.sqlite.org/c3ref/enable_load_extension.html

  The historic page about loadable extensions is:

     http://www.sqlite.org/cvstrac/wiki?p=LoadableExtensions

  The page is positively ancient (is it still experimental?  Certainly not
  new).  Various other pages from the cvstrac wiki were promoted to full
  documentation pages.  For example the one about virtual tables:

     http://www.sqlite.org/cvstrac/wiki?p=VirtualTables

  Became this:

     http://www.sqlite.org/vtab.html

  It is probably a good idea to make the old wiki pages do an HTTP redirect
  to the new correct and up to date content so programmers don't get
  confused as both show up in Google searches.


tkt_uuid: b7c8682cc17f32903f03a610bd0d35ffd3c1e6e4
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.10
resolution: Fixed
title: Incorrect result from LEFT JOIN with OR in the WHERE clause
comment:
  The following SQL demonstrates the problem:

  <blockquote><pre>
  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c, d);
  CREATE TABLE t2(x INTEGER PRIMARY KEY, y);
  CREATE TABLE t3(p INTEGER PRIMARY KEY, q);
  INSERT INTO t1 VALUES(2,3,4,5);
  INSERT INTO t1 VALUES(3,4,5,6);
  INSERT INTO t2 VALUES(2,4);
  INSERT INTO t3 VALUES(5,55);<p>
  SELECT *
    FROM t1 LEFT JOIN t2 ON y=b JOIN t3
   WHERE c==p OR d==p;
  </pre></blockquote>

  Prior to check-in [b23ae131874bc5c621f0] the query returns two rows.  But
  following that check-in, only one row is returned.  The two-row result is
  correct.

  This problem was discovered by Alan Chandler and reported on the mailing
  list as "Strange difference between sqlite 3.7.3 and 3.7.10".


tkt_uuid: 25e09aa2ab20d67a241b0164ac6818a27ea248d0
type: Feature_Request
status: Closed
subsystem: Parser
priority: Immediate
severity: Important
resolution: Rejected
title: Lint mode
comment:
  It would be really nice if there was some sort of "lint" mode so that it would be possible to point out to developers when they are doing something nonsensical, undefined or suboptimal.


tkt_uuid: 27ca74af3c083f787a1c44b11fbb7c53bdbbcf1e
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.11
resolution: Fixed
title: Nested SAVEPOINT release causes pending query abort
comment:
  The following problem was reported on the SQLite users mailing list
  by Lukas Gebauer.  See
  [http://www.mail-archive.com/sqlite-users%40sqlite.org/msg69084.html]
  for the original bug report.  Synopsis:

  <blockquote><pre><nowiki>
  Sqlite version 3.7.11 on Windows cause error in my program. Some of my
  pending statements are aborted by a nested savepoint transaction. See:

  SAVEPOINT x1;
  SELECT * FROM sometable; //just open statement and left it opened!
  SAVEPOINT x2;
  RELEASE x2;
  ... call sqlite3_step on previously opened statement - I got SQLITE_ABORT
  error, even I not see any Rollback.

  When I omit savepoint x1, then statement working well. If I omit nested
  x2 savepoint, statement working fine too.
  </nowiki></pre></blockquote>

  <hr><i>drh added on 2012-03-31 17:15:38:</i><br>
  Problem originates with check-in [9c572d424a20b0585b] on [2012-02-13]


tkt_uuid: 2a5629202f85f3d1289ca13d4ed0115382815d7b
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: trunk
resolution: Fixed
title:
  Malfunctioning interaction between a multi-term ORDER BY clause and UNIQUE index containing NULL values

comment:
  Consider:

  <verbatim>
    CREATE TABLE t1(a UNIQUE, b);
    INSERT INTO t1 VALUES(NULL, 2);
    INSERT INTO t1 VALUES(NULL, 1);
    SELECT * FROM t1 ORDER BY a, b;
  </verbatim>

  SQLite currently assumes that since the values in column "a" are "UNIQUE",
  scanning the index on column a will deliver rows in an order compatible with
  "ORDER BY a, b".

  However, since UNIQUE indexes allow duplicate NULL values, this assumption
  only holds true if there is a NOT NULL constraint on the UNIQUE column.

  <hr><i>dan added on 2012-04-20 14:58:22:</i><br>
  <verbatim>

  SQLite version 3.7.12 2012-04-20 12:02:32
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a UNIQUE, b);                                                                                         
  sqlite> INSERT INTO t1 VALUES(NULL, 2);
  sqlite> INSERT INTO t1 VALUES(NULL, 1);
  sqlite> SELECT * FROM t1 ORDER BY a, b;
  |2
  |1
  </verbatim>

  <hr><i>dan added on 2012-04-20 17:41:40:</i><br>
  Introduced here: [e53403b6].

  <hr><i>dan added on 2012-04-20 17:43:46:</i><br>
  Fixed here: [9870e4c4fe] 


tkt_uuid: 385a5b56b989ce6f65e859386f932c2c8db65b5c
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Critical
resolution: Fixed
title: A DISTINCT SELECT optimized using a UNIQUE index may allow duplicate NULL values.
comment:
  Currently, SQLite assumes that if the result-set of a SELECT DISTINCT query is collectively subject to a UNIQUE constraint, the DISTINCT qualifier is redundant.

  However, this is only true if there are no NULL values in the result set, as NULL values are considered distinct for the purposes of UNIQUE constraints, but identical for the purposes of DISTINCT processing. For example:

  <verbatim>
  SQLite version 3.7.12 2012-04-20 15:24:53
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(x, y);                   
  sqlite> INSERT INTO t1 VALUES(NULL, 'row');
  sqlite> INSERT INTO t1 VALUES(NULL, 'row');
  sqlite> SELECT DISTINCT x, y FROM t1;
  |row
  sqlite> CREATE UNIQUE INDEX i1 ON t1(x, y);
  sqlite> SELECT DISTINCT x, y FROM t1;
  |row
  |row
  sqlite> 
  </verbatim>

  <hr><i>dan added on 2012-04-20 17:27:06:</i><br>
  Introduced here: [45e581bf]

  <hr><i>dan added on 2012-04-20 17:44:18:</i><br>
  Fixed here: [7b8548b187]


tkt_uuid: bdc6bbbb381e87ffddafa2a5856da87c173e0cb1
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.11
resolution: Fixed
title: Problem with FTS aux. functions when used with OR queries with enhanced query syntax enabled
comment:
  Consider the following:
  <verbatim>
    CREATE VIRTUAL TABLE t1 USING fts3(x);
    INSERT INTO t1 VALUES('a b c');
  </verbatim>

  Then:

  <verbatim>
    SELECT snippet(t1,'/','/') FROM t1 WHERE t1 MATCH '(a AND d) OR c';
  </verbatim>

  The snippet should come back something like:

  <verbatim>
    "/a/ b /c/"
  </verbatim>

  Currently, depending on the contents of the table, sometimes it comes
  back "a b /c/". In this case the first term of the document is not
  highlighted as it was part of an expression branch that did not match
  (since "a AND d" is not true). This is incorrect.

  <hr><i>dan added on 2012-05-10 17:58:47:</i><br>
  Fixed here: [4d3e1673b3]


tkt_uuid: cfe84e86839e59e84f15bab1c08d3fdf718e86d4
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Important
resolution: Fixed
title: The ".schema" command outputs nested VIEWS in the wrong order
comment:
  While working on the crash bug of 3.7.12 in Skrooge, it was found that the
  ".schema" might output nested views in the wrong order.  Example:

  <blockquote><pre>
      CREATE TABLE t1(x);
      CREATE VIEW v2 AS SELECT x+1 AS y FROM t1;
      CREATE VIEW v1 AS SELECT y+1 FROM v2;
      .schema
  </pre></blockquote>

  In the output, the definition of v1 occurs before the definition of v2, which
  renders an unparseable schema.


tkt_uuid: c2ad16f997ee9c8ed9d3cc4bc5a3da5397a074ec
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.12
resolution: Fixed
title: Segfault on query involving deeply nested aggregate views
comment:
  Running the attached script (which consists of the schema of
  [http://www.skrooge.org | Skrooge] following by a single query)
  results in a segfault.  The original bug report can be seen
  at [https://bugs.kde.org/show_bug.cgi?id=300183] and the report
  on the SQLite mailing list is at [http://www.mail-archive.com/sqlite-users%40sqlite.org/msg70289.html]

  Bisecting suggests that the problem was inserted by
  check-in [430bb59d798286a86c351].

  <hr><i>drh added on 2012-05-21 16:54:14:</i><br>
  Simpler test case:

  <blockquote><pre>
  CREATE TABLE t1(x,y);
  SELECT max((SELECT a FROM (SELECT count(1) AS a FROM t1))) FROM t1;
  </pre></blockquote>

  <hr><i>drh added on 2012-05-21 17:14:14:</i><br>
  Another simple test case:

  <blockquote><pre>
  CREATE TABLE t1(x);
  CREATE TABLE t2(y);
  SELECT max((SELECT avg(x) FROM t2)) FROM t1;
  </pre></blockquote>


tkt_uuid: d02e1406a58ea02de21627afaea1a5acd3a2e673
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.7.14
resolution: Fixed
title: LEFT JOIN with an OR in the ON clause causes segfault
comment:
  The following SQL results in a segfault while running the SELECT statement.

  <blockquote><pre>
  CREATE TABLE t1(a,b,c);
  INSERT INTO t1 VALUES(1,2,3), (4,5,6);
  CREATE TABLE t2(d,e,f);
  INSERT INTO t2 VALUES(3,6,9), (4,8,12);
  SELECT * FROM t1 AS x LEFT JOIN t2 AS y ON (y.d=x.c) OR (y.e=x.b);
  </blockquote></pre>

  This problem was initially reported on the [http://www.mail-archive.com/sqlite-users%40sqlite.org/msg72899.html | mailing list].  The problem 
  appears to have been introduced by check-in [62678be3df35cdcb091] and first
  appeared in version 3.7.14.


tkt_uuid: bfbf38e5e9956ac69f9491e5112636ff2ecef923
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: trunk:2012-11-02
private_contact: 21acf606a926cf9bbf2ac304bb3928dcd0e489fe
resolution: Fixed
title: Segfault on a nested join
comment:
  Reported on the mailing list:

  <blockquote><pre>
  CREATE TABLE AAA (
    aaa_id       INTEGER PRIMARY KEY AUTOINCREMENT
  );
  CREATE TABLE RRR (
    rrr_id      INTEGER     PRIMARY KEY AUTOINCREMENT,
    rrr_date    INTEGER     NOT NULL,
    rrr_aaa     INTEGER
  );
  CREATE TABLE TTT (
    ttt_id      INTEGER PRIMARY KEY AUTOINCREMENT,
    target_aaa  INTEGER NOT NULL,
    source_aaa  INTEGER NOT NULL
  );
  insert into AAA (aaa_id) values (2);
  insert into TTT (ttt_id, target_aaa, source_aaa)
  values (4469, 2, 2);
  insert into TTT (ttt_id, target_aaa, source_aaa)
  values (4476, 2, 1);
  insert into RRR (rrr_id, rrr_date, rrr_aaa)
  values (0, 0, NULL);
  insert into RRR (rrr_id, rrr_date, rrr_aaa)
  values (2, 4312, 2);
  SELECT i.aaa_id,
    (SELECT sum(CASE WHEN (t.source_aaa == i.aaa_id) THEN 1 ELSE 0 END)
       FROM TTT t
    ) AS segfault
  FROM
   (SELECT curr.rrr_aaa as aaa_id
      FROM RRR curr
        -- you also can comment out the next line
        -- it causes segfault to happen after one row is outputted
        INNER JOIN AAA a ON (curr.rrr_aaa = aaa_id)
        LEFT JOIN RRR r ON (r.rrr_id <> 0 AND r.rrr_date < curr.rrr_date)
     GROUP BY curr.rrr_id
    HAVING r.rrr_date IS NULL
  ) i;
  </pre></blockquote>

  <hr><i>drh added on 2012-11-02 12:58:45:</i><br>
  Problem history:

  The query works prior to [430bb59d798286a] on 2012-04-17 in between versions 3.7.11 and 3.7.12.  In release version 3.7.12, 2012-05-14, the script
  causes a segfault in the query planner.  The query planner segfault is
  recorded by ticket [c2ad16f997ee9c8] and is fixed by check-in
  [e459c3360ba2543].  After this fix, the query reported by this ticket
  returns "misuse of aggregate: sum()".  That was the result as of
  version 3.7.13.  The current problem (a segfault in the VM while running
  the query) originates with check-in [d4cd6017c987594] on 2012-08-24, in
  between version 3.7.13 and 3.7.14.  

  <hr><i>drh added on 2012-11-02 13:51:11:</i><br>
  Simplified test case:

  <blockquote><pre>
  CREATE TABLE t1 (
    id1 INTEGER PRIMARY KEY AUTOINCREMENT,
    value1 INTEGER
  );
  INSERT INTO t1 VALUES(4469,2),(4476,1);
  CREATE TABLE t2 (
    id2 INTEGER PRIMARY KEY AUTOINCREMENT,
    value2 INTEGER
  );
  INSERT INTO t2 VALUES(0,1),(2,2);
  SELECT
   (SELECT sum(value2==xyz) FROM t2)
  FROM
   (SELECT curr.value1 as xyz
      FROM t1 AS curr LEFT JOIN t1 AS other
     GROUP BY curr.id1);
  </pre></blockquote>

  <hr><i>drh added on 2012-11-02 14:43:51:</i><br>
  Here is another simplification of the script that avoids the use of LEFT JOIN and silly constructs like a GROUP BY on a PRIMARY KEY:

  <blockquote><pre>
  CREATE TABLE t1 (
    id1 INTEGER,
    value1 INTEGER,
    x1 INTEGER
  );
  INSERT INTO t1 VALUES(4469,2,98),(4469,1,99),(4469,3,97);
  CREATE TABLE t2 (
    value2 INTEGER
  );
  INSERT INTO t2 VALUES(1);
  pragma vdbe_listing=on; pragma vdbe_trace=on;
  SELECT
   (SELECT sum(value2==xyz) FROM t2)
  FROM
   (SELECT value1 as xyz, max(x1) AS pqr
      FROM t1
     GROUP BY id1);
  </pre></blockquote>

  <hr><i>drh added on 2012-11-02 15:14:42:</i><br>
  An even simpler demonstration that omits the subquery in the FROM clause:

  <blockquote><pre>
  CREATE TABLE t1(id1, value1);
  INSERT INTO t1 VALUES(4469,2),(4469,1);
  CREATE TABLE t2 (value2);
  INSERT INTO t2 VALUES(1);
  SELECT (SELECT sum(value2=value1) FROM t2), max(value1)
    FROM t1
   GROUP BY id1;
  </pre></blockquote>


tkt_uuid: 5eaa61ea1881040b17449ca043b6f8fd9ca55dc3
type: Portability
status: Fixed
subsystem: Unknown
priority: Low
severity: Minor
foundin: 2993ca20207f8dac02f58d01e31d68c84328356a
private_contact: d8897a439aa8ad6ba5c5a703f9f6fce3a649bdf3
resolution: Fixed
title: sigbus on disk ful in WAL mode
comment:
  Attempts to prepare a query on a WAL database when the disk space is critically low result in the process killed with BUS.
  The crash happens in walIndexWriteHdr invoked from walIndexRecover:
  <verbatim>
   643 static void walIndexWriteHdr(Wal *pWal){
   [...]
   650   walChecksumBytes(1, (u8*)&pWal->hdr, nCksum, 0, pWal->hdr.aCksum);
   651   memcpy((void *)&aHdr[1], (void *)&pWal->hdr, sizeof(WalIndexHdr)); << sigbus here
  </verbatim>

  Since i don't know if I'll be able to attache a PoC after submitting this ticket, I'm adding it here in base64.

  <verbatim>
  base64 -d > mountme.bz2 << EOF
  QlpoOTFBWSZTWdl2FE4AAEd////////v7/5e/////L9v/+xv38ZSbAdOQV4FRCTR3/3s0ARNY3on
  aVG5ydncNEgTQE09IyngieU8gh6TINNPUAGmjQ2o0HpDQNNHqHqAAADTQ9RoeU0GR5Q8kBoZNqeo
  NEJpkRojCniamZNJo02oAzUNNGmyg0DIAAAAABoAAAAAANAaAAAxFE0Jij0J6hp5NR6Rk0ANAZBo
  GgAAAADQAAAAAAAAABoDQAgNDINMmmgYTQGhpk0aYEZGTQZDJkGRpkaNMg0aAGCNNBiGgAZNDJgI
  0GjQJFJRNNNDIGjTIbU0HqABoaaZDINAAAADQBiAGIGjQyNNGgAZMI0ZNDJoU3MeFxkkc3Z41hbW
  8jDkSjtOhgCTZvZDyAuQe9MlnyUJRsnBSKCqYknJpalrzfoZNLCdV70TdiIXxMRuU2IRkQJqG8QZ
  OQb+iYlGMFPlV0OmpxTALbroIwQMEAHxal5pbiPEEKHH0kN2QtWKoGkgJsW5O0ayrdAnCOCFYLWw
  lLfOMYqAyp0kAifKYYjJCJmGkTTtwu6OsOY0RUoOiYDGtSQctLi3zirl93O1IcPEMJd2yfKOo5VC
  tWwG8FR62hLdnPEziKCWNRAajF4aK+BaiAUlAZ2KQdmyNlQkNBaRNnI7pJPgEa24whyQSEgCKdFC
  /aaQiZaWUYc9gZLodGIx76jQRmC3DBQYgKCBI5zBjgkBPQAMNSLg5jVDAwp0lCg6hCZ3cBnTAKCr
  cB3YAkh1CQjQRAjhACk7EMgIYG/qJECI59jLeOZbEgNUsCyBI1VUxJGKqGBEkyVYgcIAm0kNpA2k
  Ek0adiIMAxDQmwQlHkcPEajEqCMs0KPm/J9/zgpwGR8bJkqz2cyCKk0efK1+mDyI2XAXOKDBngnh
  ufJHQ/UIOjK+VQ2Co9F59shqwue3JMARlIR+wwrBKYWf7aqVbNpnE2YHnSYYO78LCCQlXsSBlmIB
  gq+Mhwc5o4DY3koqRgDooKXZs6xZl9Pu99c/JOtbNB8swjv0DLMzBv1CaRMihk7i65cnPsysYte/
  g2qofXNK7kUHLlkQDswL2sD2Qxg18u1zVCzMcRKmYKwk+dyYVqIGgU9AWSAGFQncaikWEUsdh1cy
  DMTB1MUJJmoPKklNQgJliXgmCc48b5DvnmlACYCBG40dGAieggokSkAKv+aEPAwFF0hGpNk0JRQM
  hQpUWBdCAbFtYhjq4Qnk2IF16/kvnNfAx5nFlgNSp3At7mGEVQwmTWM2Pah6bczg7MK84eAyvC0S
  qZiDAVigjMRQUGiL+XmKkF1Sm6e/kUVALAMBBOMBKbkBCQEbxxkumARAAyffwCvK4lyRZC1cImG/
  O6CB4G0G2dmooW+OATX2TIF8lYiCl4hSdrvCgNAxQgtgsr8iCzICwdQCJPjc8hDYS67AulWQxxmw
  1WMNwqrviGcCT9CYWMaONSepUu5bLC9KfhepOKSBVBmQ8AZBSyB1ntos5HzRZMwLjGu6tPVUQKOd
  ozKDukqH+aYanaMEJAE+T6suVBNc/fNGBlP1+ZqyEaJCdpvPRVi/VJMHdoKy2QHNOxo3k8kUx7rg
  w8E49w6q6osXLYPmkQ2hELLO5TWp7D8RhybGAo6WXzt66MIRIkZ1kgghKRKYMAgB7YcebZbV7ada
  B/Asgf4nIUJ8iKZ1qbCYDBkg7H8sTf6oDqczF/X79LiIqxb1boNG662tp/Vz7JfwSCL6w4+2fqk3
  ym9/ynQJUzxwAZf7EwajzMXWBxQgiIjYJJ0YJATYEIr+KoYlEwFWSnnjkI3piIP1ycNM6496w1SD
  V/Xf8tkMAB4Ffbf/i7kinChIbLsKJwA=
  EOF

  Then:
  bunzip2 mountme.bz2
  mount -o loop mountme /mnt

  Finally:
  # sqlite3 /mnt/db.sqlite3 
  SQLite version 3.7.14.1 2012-10-04 19:37:12
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> .tables
  Bus error
  </verbatim>

  <hr><i>drh added on 2012-11-13 01:28:57:</i><br>
  Unable to recreate the malfunction here.  In fact, gdb says that the
  walIndexWriteHdr() function never gets called when following the steps
  outlined in the original bug report (which is exactly what you would
  expect if you are only doing a query.)

  <hr><i>anonymous added on 2012-11-13 09:02:21:</i><br>
  Are you running sqlite3/gdb as root?

  <hr><i>drh added on 2012-11-13 09:28:13:</i><br>
  Yes, the sqlite3 shell was run as run.  I also repeated the experiment multiple times with valgrind, for what it is worth.  No issues observed.

  Furthermore, I tried creating write-ahead log files that needed to be recovered (since the original problem statement said that the segfault occurred during recovery) and recover them, as root, with zero space left on the device.  Still no problems.

  The segfault occurs on a line that is attempting to write into a newly allocated mmap-ed file (the db.sqlite-shm file, specifically).  It appears that Linux will allow disk space for mmap-ed files to be overcommitted.  That is to say, based on my experiments, you can mmap more space than you have on disk and the mmap() call still works.  Perhaps you system is configured in some way different from mine (perhaps it is also under memory pressure) so that the mmap-ed region is becoming unmapped somehow?

  <hr><i>drh added on 2012-11-13 10:34:17:</i><br>
  Able to reproduce the problem now....

  <hr><i>anonymous added on 2012-11-13 12:14:38:</i><br>
  Confirmed resolved. Thanks a lot for the very quick fix!


tkt_uuid: 01b785eefa32b6b7322318a94d61e9b0171e12c0
type: Support_Request
status: Closed
subsystem: Shell
priority: Immediate
severity: Important
foundin: 3.7.14.1
private_contact: 5622daa5fb452498d0b3d05600106fca2bc3174f
resolution: External_Bug
title: sqlite3.exe - wrong parameter charset
comment:
  I wish to run:
  sqlite3.exe some_database_name.db3

  While some_database_name.db3 contains some non-ascii chars, I am in trouble! sqlite3.exe expects this parameter in UTF-8. It is not usual, and I am unable to write it by keyboard in windows commandline shell.

  I am expecting all program parameters to be in native windows encoding, and conversion from native windows encoding to UTF-8 is made inside sqlite3.exe.

  Then I can write non-ascii path in commandline shell by OEM encoding, then I can run it by CreateprocessA Win32API as ANSI encoding or by CreateProcessW Win32API as widechar.


tkt_uuid: f46ca03b55ae0ecab698c623031abbb550e1f405
type: Code_Defect
status: Closed
subsystem: Extensions
priority: Immediate
severity: Important
foundin: 3.7.14.1
private_contact: 7775aa316ff01dfe638bbf525a453db28dc737a8
resolution: Overcome_By_Events
title: The order of two operations in asyncLock()
comment:
  In file sqlite3async.c, the function asyncLock(), line 888
  Why set the lock.eLock and lock.eAsyncLock before getFileLock() is called?
  If getFileLock() failed, why not reset the eLock and eAsyncLock?
  It looks like if the first asyncLock() called failed, the second will succeed without any check?


tkt_uuid: ae3c5670b65f049e09ff0a0fb913bdcf589ab0b7
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Important
foundin: trunk
resolution: Fixed
title: Bug caused by factoring of constants in trigger programs
comment:
  Depending on the indexes on the table, SQLite may transform queries 
  similar to the following:

  <verbatim>
    CREATE TABLE t1(a, b, c);
    SELECT * FROM t1 WHERE
      (a IN (sub-query-a) OR b IN (sub-query-b)) AND c IN (sub-query-c);
  </verbatim>

  to

  <verbatim>
    SELECT * FROM t1 WHERE a IN (sub-query-a) AND c IN (sub-query-c)
    UNION
    SELECT * FROM t1 WHERE b IN (sub-query-b) AND c IN (sub-query-c)
  </verbatim>

  Separate vdbe code is generated for each copy of sub-query-c.

  If (sub-query-a) returns 0 rows, the first copy of (sub-query-c) will never be executed. And so the code generated for the second copy of (sub-query-c) must not assume that any constants stored in registers by the first copy are in fact already present when the second copy of (sub-query-c) is executed. And usually, SQLite does not assume this. However, due to a bug, it may if a SELECT similar to the above is included in a trigger program.

  Before the bug was fixed, the following SQL script did not insert a row into t5. Even though the INSERT INTO SELECT should have copied the row from t1 to t5, it did not because the constant 1234567 was affected by the bug described above.

  <verbatim>
    CREATE TABLE t1(a, b, c);
    CREATE INDEX i1 ON t1(a, c);
    CREATE INDEX i2 ON t1(b, c);
    INSERT INTO t1 VALUES(1, 2, 3);

    CREATE TABLE t2(e, f);
    CREATE INDEX i3 ON t2(e);
    INSERT INTO t2 VALUES(1234567, 3);

    CREATE TABLE empty(x);
    CREATE TABLE not_empty(x);
    INSERT INTO not_empty VALUES(2);

    CREATE TABLE t4(x);
    CREATE TABLE t5(g, h, i);

    CREATE TRIGGER trig BEFORE INSERT ON t4 BEGIN
      INSERT INTO t5 SELECT * FROM t1 WHERE
          (a IN (SELECT x FROM empty) OR b IN (SELECT x FROM not_empty))
          AND c IN (SELECT f FROM t2 WHERE e=1234567);
    END;

    INSERT INTO t4 VALUES(0);
  </verbatim>

  <hr><i>dan added on 2012-12-06 19:42:16:</i><br>
  Now fixed here: [79ef8e3c77]


tkt_uuid: d6b36be38a06c0525731e6357125e6a35ecc4b79
type: Code_Defect
status: Closed
subsystem: Code_Generator
priority: Immediate
severity: Critical
foundin: trunk
resolution: Fixed
title: Sub-select in LIMIT clause may access db without locks.
comment:
  This problem affects a SELECT statement only if all the following are true:

  <ul>
    <li> The FROM clause consists of a single sub-select.
    <li> The FROM clause sub-select cannot be flattened into its parent.
    <li> Either the LIMIT or OFFSET clause of the outer SELECT is also a
         sub-select.
  </ul>

  When SQLite processes a SELECT statement for which the FROM clause contains a
  single sub-select that is not eligible for the flattening optimization, it
  creates a co-routine that returns one row each time it is invoked to
  implement the sub-select.

  The vdbe code is:

  <ul>
     <li> Jump to start of main routine,
     <li> Body of co-routine,
     <li> Main routine (invokes sub-routine).
     <li> VerifyCookie routine.
  </ul>

  Because of the way the SQL compiler works, the VerifyCookie routine is
  currently being invoked the first time the co-routine is invoked, not by the
  start of the main routine. If the main routine attempts to access the database
  before the co-routine is invoked, then it is an error. This may occur if either
  the LIMIT or the OFFSET expression of the main SELECT are themselves
  sub-selects that access the database. For example:

  <verbatim>
    CREATE TABLE abc(a, b, c);
    CREATE TABLE def(a, b, c);

    SELECT * FROM (SELECT * FROM def LIMIT 1 OFFSET 1) LIMIT (SELECT 20 FROM abc)
  </verbatim>


tkt_uuid: df678d738adb24e9d7f5beff3a89928161c072fc
type: Code_Defect
status: Closed
subsystem: Pager
priority: Immediate
severity: Critical
resolution: Fixed
title: Cache corruption following IO error on systems supporting atomic-write
comment:
  If SQLite is compiled with the SQLITE_ENABLE_ATOMIC_WRITE symbol defined and a custom VFS indicates that it is supported, SQLite writes transactions that modify a single database page on disk without creating a journal file. This is safe, since the system guarantees that single write required is atomic.

  For such systems, SQLite creates an "in-memory journal" when a write transaction is opened. After the second database page is modified within the cache, a real journal file is created on disk and the contents of the in-memory journal file written to it. Processing then continues as per normal. Or, if the transaction never modifies a second page, the journal file is never created on disk. 

  If the user opens a write transaction and then issues a ROLLBACK command after modifying a single page, the contents of the in-memory journal are used to restore the page-cache to its original state.

  However, due to a bug, if an IO or disk full error occurs while transferring the contents of the in-memory journal to disk, SQLite is not using the contents of the in-memory journal to restore the in-memory cache. Instead, it is attempting to use the contents of the file on disk (which was not written correctly - if at all). This means the single page that was modified in the cache is not being rolled back - cache corruption.

  For example, the INSERT statement in the SQL below writes to two database pages - one in the table b-tree and the other in the index. Internally, SQLite first modifies the table and stores the old page image in the in-memory journal. Before modifying the index, it attempts to write the contents of the in-memory journal out to disk. If it encounters a disk-full error while doing so, the subsequent rollback does not restore the contents of the table (due to the bug), leaving the connection with a corrupt cache.

  <verbatim>
    CREATE TABLE t1(a, b);
    CREATE INDEX i1 ON t1(a, b);
    INSERT INTO t1 VALUES(1, 2);
  </verbatim>


tkt_uuid: 0cfd98ee201a2aa8f3f494bcc90d2977dc4d1990
type: Code_Defect
status: Closed
subsystem: Pager
priority: Immediate
severity: Important
foundin: 3.7.15
resolution: Fixed
title: Potential corruption of backup db when backing up a db last written by sqlite<3.7.0
comment:
  Versions of sqlite 3.7.0 and newer store the size of the database image in the database file header. Earlier versions do not. When a new version writes to the db, it updates the database size field and sets a flag in the header to indicate that the last version to write the db updated the db size field. An earlier version does not update the db size field, but does clear the flag.

  If the flag is set, SQLite trusts the db size field in the header. So if it is set incorrectly, the database is considered corrupt.

  The current problem comes about as follows:

  <ol>
    <li>A 'new' version of sqlite writes to the database file, setting the db size field.
    <li>An 'old' version of sqlite writes the db and changes the file size. This means the db size field is now invalid, but the flag is cleared so the db is not corrupt.
    <li>A 'new' version of sqlite does a backup of the db. In the backup, the db size field remains unchanged, but the "written by a new version" flag is set. So the backup is now corrupted.
  </ol>

  Note: The "flag" mentioned above is not actually a flag, it is a second copy of the change counter that is only updated by post-3.7.0 versions of sqlite. The flag is considered set if that second copy matches the first.


tkt_uuid: c997b11c4d53a2ee9983111d0c676dd7d4f01ec7
type: Code_Defect
status: Fixed
subsystem: Code_Generator
priority: Immediate
severity: Important
resolution: Fixed
title: ORDER BY clause ignored in 3-way join query
comment:
  <verbatim>
  $ ./sqlite3 
  SQLite version 3.7.16 2013-01-08 12:48:10
  Enter ".help" for instructions
  Enter SQL statements terminated with a ";"
  sqlite> CREATE TABLE t1(a INTEGER PRIMARY KEY);
  sqlite> CREATE TABLE t2(b INTEGER PRIMARY KEY, c INTEGER);
  sqlite> CREATE TABLE t3(d INTEGER);
  sqlite> 
  sqlite> INSERT INTO t1 VALUES(1);
  sqlite> INSERT INTO t1 VALUES(2);
  sqlite> INSERT INTO t1 VALUES(3);
  sqlite> 
  sqlite> INSERT INTO t2 VALUES(3, 1);
  sqlite> INSERT INTO t2 VALUES(4, 2);
  sqlite> INSERT INTO t2 VALUES(5, 3);
  sqlite> 
  sqlite> INSERT INTO t3 VALUES(4);
  sqlite> INSERT INTO t3 VALUES(3);
  sqlite> INSERT INTO t3 VALUES(5);
  sqlite>
  sqlite> SELECT t1.a FROM t1, t2, t3 WHERE t1.a=t2.c AND t2.b=t3.d ORDER BY t1.a;
  2
  1
  3
  </verbatim>

  Problem introduced by [956e4d7f89].

  First reported on the mailing list: http://www.mail-archive.com/sqlite-users@sqlite.org/msg74717.html


tkt_uuid: ba82a4a41eac1f38eebacf7d28b6b2273a9c497e
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Severe
foundin: 3.7.16.2
resolution: Fixed
title: Query optimizer removes ORDER BY when it is needed
comment:
  <hr><i>drh claiming to be  added on 2013-04-22 22:29:50:</i><br>
  Regression tests added to TH3 in the where33.test module.


tkt_uuid: f67b41381acce9a881fb081c4a20ecbeb2429b0d
type: Code_Defect
status: Fixed
subsystem: Unknown
priority: Immediate
severity: Critical
resolution: Fixed
title:
  "INSERT INTO tbl SELECT * FROM tbl2" statement may modify data if ALTER TABLE ADD COLUMN has been used on tbl2.


tkt_uuid: 8a2adec166701e42e6f5dab214f80a1340864bb4
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.8.11 - 3.9.1
resolution: Fixed
title:
  Problem creating automatic-indexes on sub-queries in the FROM clause of correlated scalar sub-queries


tkt_uuid: 50d2a6c292e3c391000081973cb23dcd2659cbc0
type: Code_Defect
status: Closed
subsystem: Unknown
priority: Immediate
severity: Critical
foundin: 3.26.0
resolution: Fixed
title:
  Foreign key malfunction when parent and child are the same WITHOUT ROWID table and parent key is not also PRIMARY KEY


