Skip to content

Commit

Permalink
Merge pull request #8 from wojas/upgrade-lmdb-0-9-28
Browse files Browse the repository at this point in the history
Upgrade LMDB C lib from 0.9.19 to 0.9.28
  • Loading branch information
wojas authored Apr 20, 2021
2 parents 58afa91 + c6b9a86 commit 33d501a
Show file tree
Hide file tree
Showing 6 changed files with 615 additions and 160 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
/lmdb/openldap

/bin/
/tmp/

# Test databases and benchmark data
testdata/
Expand All @@ -29,3 +30,4 @@ _cgo_export.*
_testmain.go

*.exe

262 changes: 262 additions & 0 deletions CHANGES.lmdb.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
LMDB 0.9 Change Log

LMDB 0.9.28 Release (2021/02/04)
ITS#8662 add -a append option to mdb_load

LMDB 0.9.27 Release (2020/10/26)
ITS#9376 fix repeated DUPSORT cursor deletes

LMDB 0.9.26 Release (2020/08/11)
ITS#9278 fix robust mutex cleanup for FreeBSD

LMDB 0.9.25 Release (2020/01/30)
ITS#9068 fix mdb_dump/load backslashes in printable content
ITS#9118 add MAP_NOSYNC for FreeBSD
ITS#9155 free mt_spill_pgs in non-nested txn on end

LMDB 0.9.24 Release (2019/07/24)
ITS#8969 Tweak mdb_page_split
ITS#8975 WIN32 fix writemap set_mapsize crash
ITS#9007 Fix loose pages in WRITEMAP

LMDB 0.9.23 Release (2018/12/19)
ITS#8756 Fix loose pages in dirty list
ITS#8831 Fix mdb_load flag init
ITS#8844 Fix mdb_env_close in forked process
Documentation
ITS#8857 mdb_cursor_del doesn't invalidate cursor
ITS#8908 GET_MULTIPLE etc don't change passed in key

LMDB 0.9.22 Release (2018/03/22)
Fix MDB_DUPSORT alignment bug (ITS#8819)
Fix regression with new db from 0.9.19 (ITS#8760)
Fix liblmdb to build on Solaris (ITS#8612)
Fix delete behavior with DUPSORT DB (ITS#8622)
Fix mdb_cursor_get/mdb_cursor_del behavior (ITS#8722)

LMDB 0.9.21 Release (2017/06/01)
Fix xcursor after cursor_del (ITS#8622)

LMDB 0.9.20 (Withdrawn)
Fix mdb_load with escaped plaintext (ITS#8558)
Fix mdb_cursor_last / mdb_put interaction (ITS#8557)

LMDB 0.9.19 Release (2016/12/28)
Fix mdb_env_cwalk cursor init (ITS#8424)
Fix robust mutexes on Solaris 10/11 (ITS#8339)
Tweak Win32 error message buffer
Fix MDB_GET_BOTH on non-dup record (ITS#8393)
Optimize mdb_drop
Fix xcursors after mdb_cursor_del (ITS#8406)
Fix MDB_NEXT_DUP after mdb_cursor_del (ITS#8412)
Fix mdb_cursor_put resetting C_EOF (ITS#8489)
Fix mdb_env_copyfd2 to return EPIPE on SIGPIPE (ITS#8504)
Fix mdb_env_copy with empty DB (ITS#8209)
Fix behaviors with fork (ITS#8505)
Fix mdb_dbi_open with mainDB cursors (ITS#8542)
Fix robust mutexes on kFreeBSD (ITS#8554)
Fix utf8_to_utf16 error checks (ITS#7992)
Fix F_NOCACHE on MacOS, error is non-fatal (ITS#7682)
Build
Make shared lib suffix overridable (ITS#8481)
Documentation
Cleanup doxygen nits
Note reserved vs actual mem/disk usage


LMDB 0.9.18 Release (2016/02/05)
Fix robust mutex detection on glibc 2.10-11 (ITS#8330)
Fix page_search_root assert on FreeDB (ITS#8336)
Fix MDB_APPENDDUP vs. rewrite(single item) (ITS#8334)
Fix mdb_copy of large files on Windows
Fix subcursor move after delete (ITS#8355)
Fix mdb_midl_shirnk off-by-one (ITS#8363)
Check for utf8_to_utf16 failures (ITS#7992)
Catch strdup failure in mdb_dbi_open
Build
Additional makefile var tweaks (ITS#8169)
Documentation
Add Getting Started page
Update WRITEMAP description


LMDB 0.9.17 Release (2015/11/30)
Fix ITS#7377 catch calloc failure
Fix ITS#8237 regression from ITS#7589
Fix ITS#8238 page_split for DUPFIXED pages
Fix ITS#8221 MDB_PAGE_FULL on delete/rebalance
Fix ITS#8258 rebalance/split assert
Fix ITS#8263 cursor_put cursor tracking
Fix ITS#8264 cursor_del cursor tracking
Fix ITS#8310 cursor_del cursor tracking
Fix ITS#8299 mdb_del cursor tracking
Fix ITS#8300 mdb_del cursor tracking
Fix ITS#8304 mdb_del cursor tracking
Fix ITS#7771 fakepage cursor tracking
Fix ITS#7789 ensure mapsize >= pages in use
Fix ITS#7971 mdb_txn_renew0() new reader slots
Fix ITS#7969 use __sync_synchronize on non-x86
Fix ITS#8311 page_split from update_key
Fix ITS#8312 loose pages in nested txn
Fix ITS#8313 mdb_rebalance dummy cursor
Fix ITS#8315 dirty_room in nested txn
Fix ITS#8323 dirty_list in nested txn
Fix ITS#8316 page_merge cursor tracking
Fix ITS#8321 cursor tracking
Fix ITS#8319 mdb_load error messages
Fix ITS#8320 mdb_load plaintext input
Added mdb_txn_id() (ITS#7994)
Added robust mutex support
Miscellaneous cleanup/simplification
Build
Create install dirs if needed (ITS#8256)
Fix ThreadProc decl on Win32/MSVC (ITS#8270)
Added ssize_t typedef for MSVC (ITS#8067)
Use ANSI apis on Windows (ITS#8069)
Use O_SYNC if O_DSYNC,MDB_DSYNC are not defined (ITS#7209)
Allow passing AR to make (ITS#8168)
Allow passing mandir to make install (ITS#8169)

LMDB 0.9.16 Release (2015/08/14)
Fix cursor EOF bug (ITS#8190)
Fix handling of subDB records (ITS#8181)
Fix mdb_midl_shrink() usage (ITS#8200)

LMDB 0.9.15 Release (2015/06/19)
Fix txn init (ITS#7961,#7987)
Fix MDB_PREV_DUP (ITS#7955,#7671)
Fix compact of empty env (ITS#7956)
Fix mdb_copy file mode
Fix mdb_env_close() after failed mdb_env_open()
Fix mdb_rebalance collapsing root (ITS#8062)
Fix mdb_load with large values (ITS#8066)
Fix to retry writes on EINTR (ITS#8106)
Fix mdb_cursor_del on empty DB (ITS#8109)
Fix MDB_INTEGERDUP key compare (ITS#8117)
Fix error handling (ITS#7959,#8157,etc.)
Fix race conditions (ITS#7969,7970)
Added workaround for fdatasync bug in ext3fs
Build
Don't use -fPIC for static lib
Update .gitignore (ITS#7952,#7953)
Cleanup for "make test" (ITS#7841), "make clean", mtest*.c
Misc. Android/Windows cleanup
Documentation
Fix MDB_APPEND doc
Fix MDB_MAXKEYSIZE doc (ITS#8156)
Fix mdb_cursor_put,mdb_cursor_del EACCES description
Fix mdb_env_sync(MDB_RDONLY env) doc (ITS#8021)
Clarify MDB_WRITEMAP doc (ITS#8021)
Clarify mdb_env_open doc
Clarify mdb_dbi_open doc

LMDB 0.9.14 Release (2014/09/20)
Fix to support 64K page size (ITS#7713)
Fix to persist decreased as well as increased mapsizes (ITS#7789)
Fix cursor bug when deleting last node of a DUPSORT key
Fix mdb_env_info to return FIXEDMAP address
Fix ambiguous error code from writing to closed DBI (ITS#7825)
Fix mdb_copy copying past end of file (ITS#7886)
Fix cursor bugs from page_merge/rebalance
Fix to dirty fewer pages in deletes (mdb_page_loose())
Fix mdb_dbi_open creating subDBs (ITS#7917)
Fix mdb_cursor_get(_DUP) with single value (ITS#7913)
Fix Windows compat issues in mtests (ITS#7879)
Add compacting variant of mdb_copy
Add BigEndian integer key compare code
Add mdb_dump/mdb_load utilities

LMDB 0.9.13 Release (2014/06/18)
Fix mdb_page_alloc unlimited overflow page search
Documentation
Re-fix MDB_CURRENT doc (ITS#7793)
Fix MDB_GET_MULTIPLE/MDB_NEXT_MULTIPLE doc

LMDB 0.9.12 Release (2014/06/13)
Fix MDB_GET_BOTH regression (ITS#7875,#7681)
Fix MDB_MULTIPLE writing multiple keys (ITS#7834)
Fix mdb_rebalance (ITS#7829)
Fix mdb_page_split (ITS#7815)
Fix md_entries count (ITS#7861,#7828,#7793)
Fix MDB_CURRENT (ITS#7793)
Fix possible crash on Windows DLL detach
Misc code cleanup
Documentation
mdb_cursor_put: cursor moves on error (ITS#7771)


LMDB 0.9.11 Release (2014/01/15)
Add mdb_env_set_assert() (ITS#7775)
Fix: invalidate txn on page allocation errors (ITS#7377)
Fix xcursor tracking in mdb_cursor_del0() (ITS#7771)
Fix corruption from deletes (ITS#7756)
Fix Windows/MSVC build issues
Raise safe limit of max MDB_MAXKEYSIZE
Misc code cleanup
Documentation
Remove spurious note about non-overlapping flags (ITS#7665)

LMDB 0.9.10 Release (2013/11/12)
Add MDB_NOMEMINIT option
Fix mdb_page_split() again (ITS#7589)
Fix MDB_NORDAHEAD definition (ITS#7734)
Fix mdb_cursor_del() positioning (ITS#7733)
Partial fix for larger page sizes (ITS#7713)
Fix Windows64/MSVC build issues

LMDB 0.9.9 Release (2013/10/24)
Add mdb_env_get_fd()
Add MDB_NORDAHEAD option
Add MDB_NOLOCK option
Avoid wasting space in mdb_page_split() (ITS#7589)
Fix mdb_page_merge() cursor fixup (ITS#7722)
Fix mdb_cursor_del() on last delete (ITS#7718)
Fix adding WRITEMAP on existing env (ITS#7715)
Fix nested txns (ITS#7515)
Fix mdb_env_copy() O_DIRECT bug (ITS#7682)
Fix mdb_cursor_set(SET_RANGE) return code (ITS#7681)
Fix mdb_rebalance() cursor fixup (ITS#7701)
Misc code cleanup
Documentation
Note that by default, readers need write access


LMDB 0.9.8 Release (2013/09/09)
Allow mdb_env_set_mapsize() on an open environment
Fix mdb_dbi_flags() (ITS#7672)
Fix mdb_page_unspill() in nested txns
Fix mdb_cursor_get(CURRENT|NEXT) after a delete
Fix mdb_cursor_get(DUP) to always return key (ITS#7671)
Fix mdb_cursor_del() to always advance to next item (ITS#7670)
Fix mdb_cursor_set(SET_RANGE) for tree with single page (ITS#7681)
Fix mdb_env_copy() retry open if O_DIRECT fails (ITS#7682)
Tweak mdb_page_spill() to be less aggressive
Documentation
Update caveats since mdb_reader_check() added in 0.9.7

LMDB 0.9.7 Release (2013/08/17)
Don't leave stale lockfile on failed RDONLY open (ITS#7664)
Fix mdb_page_split() ref beyond cursor depth
Fix read txn data race (ITS#7635)
Fix mdb_rebalance (ITS#7536, #7538)
Fix mdb_drop() (ITS#7561)
Misc DEBUG macro fixes
Add MDB_NOTLS envflag
Add mdb_env_copyfd()
Add mdb_txn_env() (ITS#7660)
Add mdb_dbi_flags() (ITS#7661)
Add mdb_env_get_maxkeysize()
Add mdb_env_reader_list()/mdb_env_reader_check()
Add mdb_page_spill/unspill, remove hard txn size limit
Use shorter names for semaphores (ITS#7615)
Build
Fix install target (ITS#7656)
Documentation
Misc updates for cursors, DB handles, data lifetime

LMDB 0.9.6 Release (2013/02/25)
Many fixes/enhancements

LMDB 0.9.5 Release (2012/11/30)
Renamed from libmdb to liblmdb
Many fixes/enhancements
51 changes: 51 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,57 @@

## v1.9.0-dev

First release of this PowerDNS/lmdb-go fork.

- Renamed module from `github.com/bmatsuo/lmdb-go` to `github.com/PowerDNS/lmdb-go` (#3, PR #7)
- Add `go.mod` and fix tests and Travis CI on recent Go versions (#5, PR #6)
- Fix: Cursor.Put would write "\x00" instead of an empty value (#1, PR #2)
- lmdb: Update LMDB C library to version 0.9.28 (#4).

```
LMDB 0.9.28 Release (2021/02/04)
ITS#8662 add -a append option to mdb_load
LMDB 0.9.27 Release (2020/10/26)
ITS#9376 fix repeated DUPSORT cursor deletes
LMDB 0.9.26 Release (2020/08/11)
ITS#9278 fix robust mutex cleanup for FreeBSD
LMDB 0.9.25 Release (2020/01/30)
ITS#9068 fix mdb_dump/load backslashes in printable content
ITS#9118 add MAP_NOSYNC for FreeBSD
ITS#9155 free mt_spill_pgs in non-nested txn on end
LMDB 0.9.24 Release (2019/07/24)
ITS#8969 Tweak mdb_page_split
ITS#8975 WIN32 fix writemap set_mapsize crash
ITS#9007 Fix loose pages in WRITEMAP
LMDB 0.9.23 Release (2018/12/19)
ITS#8756 Fix loose pages in dirty list
ITS#8831 Fix mdb_load flag init
ITS#8844 Fix mdb_env_close in forked process
Documentation
ITS#8857 mdb_cursor_del doesn't invalidate cursor
ITS#8908 GET_MULTIPLE etc don't change passed in key
LMDB 0.9.22 Release (2018/03/22)
Fix MDB_DUPSORT alignment bug (ITS#8819)
Fix regression with new db from 0.9.19 (ITS#8760)
Fix liblmdb to build on Solaris (ITS#8612)
Fix delete behavior with DUPSORT DB (ITS#8622)
Fix mdb_cursor_get/mdb_cursor_del behavior (ITS#8722)
LMDB 0.9.21 Release (2017/06/01)
Fix xcursor after cursor_del (ITS#8622)
LMDB 0.9.20 (Withdrawn)
Fix mdb_load with escaped plaintext (ITS#8558)
Fix mdb_cursor_last / mdb_put interaction (ITS#8557)
```


Changes predating the PowerDNS fork (up to 2017):

- Fix unsafe threading behavior in benchmarks (bmatsuo/lmdb-go#101)
Expand Down
16 changes: 10 additions & 6 deletions lmdb/lmdb.h
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@
*
* @author Howard Chu, Symas Corporation.
*
* @copyright Copyright 2011-2016 Howard Chu, Symas Corp. All rights reserved.
* @copyright Copyright 2011-2021 Howard Chu, Symas Corp. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted only as authorized by the OpenLDAP
Expand Down Expand Up @@ -200,7 +200,7 @@ typedef int mdb_filehandle_t;
/** Library minor version */
#define MDB_VERSION_MINOR 9
/** Library patch version */
#define MDB_VERSION_PATCH 19
#define MDB_VERSION_PATCH 28

/** Combine args a,b,c into a single integer for easy version comparisons */
#define MDB_VERINT(a,b,c) (((a) << 24) | ((b) << 16) | (c))
Expand All @@ -210,7 +210,7 @@ typedef int mdb_filehandle_t;
MDB_VERINT(MDB_VERSION_MAJOR,MDB_VERSION_MINOR,MDB_VERSION_PATCH)

/** The release date of this library version */
#define MDB_VERSION_DATE "December 28, 2016"
#define MDB_VERSION_DATE "February 4, 2020"

/** A stringifier for the version info */
#define MDB_VERSTR(a,b,c,d) "LMDB " #a "." #b "." #c ": (" d ")"
Expand Down Expand Up @@ -370,7 +370,7 @@ typedef enum MDB_cursor_op {
MDB_GET_BOTH, /**< Position at key/data pair. Only for #MDB_DUPSORT */
MDB_GET_BOTH_RANGE, /**< position at key, nearest data. Only for #MDB_DUPSORT */
MDB_GET_CURRENT, /**< Return key/data at current cursor position */
MDB_GET_MULTIPLE, /**< Return key and up to a page of duplicate data items
MDB_GET_MULTIPLE, /**< Return up to a page of duplicate data items
from current cursor position. Move cursor to prepare
for #MDB_NEXT_MULTIPLE. Only for #MDB_DUPFIXED */
MDB_LAST, /**< Position at last key/data item */
Expand All @@ -379,7 +379,7 @@ typedef enum MDB_cursor_op {
MDB_NEXT, /**< Position at next data item */
MDB_NEXT_DUP, /**< Position at next data item of current key.
Only for #MDB_DUPSORT */
MDB_NEXT_MULTIPLE, /**< Return key and up to a page of duplicate data items
MDB_NEXT_MULTIPLE, /**< Return up to a page of duplicate data items
from next cursor position. Move cursor to prepare
for #MDB_NEXT_MULTIPLE. Only for #MDB_DUPFIXED */
MDB_NEXT_NODUP, /**< Position at first data item of next key */
Expand All @@ -390,7 +390,7 @@ typedef enum MDB_cursor_op {
MDB_SET, /**< Position at specified key */
MDB_SET_KEY, /**< Position at specified key, return key + data */
MDB_SET_RANGE, /**< Position at first key greater than or equal to specified key. */
MDB_PREV_MULTIPLE /**< Position at previous page and return key and up to
MDB_PREV_MULTIPLE /**< Position at previous page and return up to
a page of duplicate data items. Only for #MDB_DUPFIXED */
} MDB_cursor_op;

Expand Down Expand Up @@ -1510,6 +1510,10 @@ int mdb_cursor_put(MDB_cursor *cursor, MDB_val *key, MDB_val *data,
/** @brief Delete current key/data pair
*
* This function deletes the key/data pair to which the cursor refers.
* This does not invalidate the cursor, so operations such as MDB_NEXT
* can still be used on it.
* Both MDB_NEXT and MDB_GET_CURRENT will return the same record after
* this operation.
* @param[in] cursor A cursor handle returned by #mdb_cursor_open()
* @param[in] flags Options for this operation. This parameter
* must be set to 0 or one of the values described here.
Expand Down
Loading

0 comments on commit 33d501a

Please sign in to comment.