1
0
Commit Graph

50 Commits

Author SHA1 Message Date
ThomasV
015648c4e5 Move get_dict method from wallet_db to json_db.
Define overloaded methods: _convert_dict _convert_values
2022-11-03 12:41:49 +01:00
bitromortac
6915e3cb10 lnpeer+wallet: use channel type for channel open
* channel_type is put into storage, serialized as int and
  deserialized as ChannelType
* check for static_remotekey is done via channel type
2022-01-20 16:47:48 +01:00
ThomasV
eadd1bebb2 Do not convert StoredDict keys to str.
Instead, convert json keys to int when storage is read.
2021-09-20 14:49:59 +02:00
Malcolm Smith
67ae678137 storage/db: use faster JSON encoder settings when wallet is encrypted
The standard json module has an optimized C encoder, but that doesn't
currently support indentation. So if you request indentation, it falls
back on the slower Python encoder.

Readability doesn't matter for encrypted wallets, so this disables
indentation when the wallet is encrypted.

-----

based on b2399b6a3e

For a large encrypted wallet, compare:
before change:
JsonDB.dump 1.3153 sec
zlib.compress 1.281 sec
ECPubkey.encrypt_message 0.1744 sec

after change:
JsonDB.dump 0.5059 sec
zlib.compress 1.3120 sec
ECPubkey.encrypt_message 0.1630 sec

Co-authored-by: SomberNight <somber.night@protonmail.com>
2021-01-06 21:14:56 +01:00
SomberNight
a7199696d3 json_db: exempt keystore from StoredDict conversion
The keystore logic would need to be significantly changed to nicely
interoperate with StoredDict/json_db logic.
(just see KeyStore.__init__() and KeyStore.dump())
For now we exempt the keystore from the recursive StoredDict conversion, as
it is a smaller change that is also easier to review for correctness.

fixes #6066
fixes #6401

also reverts 2d3c2eeea9 (which was an even hackier workaround for #6066)
2020-09-04 16:11:01 +02:00
SomberNight
d5f368c584 LN invoices: support msat precision
fixes #6250
2020-06-22 22:48:13 +02:00
SomberNight
9a70b79eea follow-up prev: try to handle json db int key madness :/ 2020-03-17 20:32:27 +01:00
ThomasV
e8ee4250d9 Do not save new channels before they are added to lnworker 2020-02-24 09:44:35 +01:00
ThomasV
dbceed2647 Restructure wallet storage:
- Perform json deserializations in wallet_db
 - use StoredDict class that keeps tracks of its modifications
2020-02-04 13:35:58 +01:00
ThomasV
7507942b7a (minor) json_db: add file header, fix formatting 2020-02-04 12:43:04 +01:00
ThomasV
63963323be storage: take the DB lock when writing to disk. 2020-02-03 17:08:34 +01:00
ThomasV
149cd9598a Separate JsonDB and WalletDB 2020-02-03 12:36:07 +01:00
ThomasV
aa51df0a1a Use attr.s for Feeupdates and Outpoints
Storage upgrade to version 23
2020-02-01 16:45:19 +01:00
SomberNight
18c6451518 json_db: only deserialize transactions on-demand 2020-01-19 05:49:12 +01:00
SomberNight
d2f132738a wallet: only select mature coins by default
this is a regression from #5721

Removed the `TxInput.is_coinbase` method as I think it is a confusing API,
instead we now have `TxInput.is_coinbase_input` and `TxInput.is_coinbase_output`.

related #5872
2020-01-02 00:43:49 +01:00
SomberNight
72491bdf18 synchronizer: request tx from server if we only have partial local tx
Note that there is a slight distinction between
`not tx.is_complete()` and `isinstance(tx, PartialTransaction)`,
which is that technically you can have a PSBT that is already complete
but was not yet converted to a standard bitcoin tx.
2019-12-16 21:15:20 +01:00
SomberNight
01fc048484 CLI: properly auto-upgrade storage when needed even if storage-encrypted
previously commands would error if user had an encrypted storage that needed upgrading
2019-12-15 20:12:51 +01:00
ThomasV
2c6a1f55fb Merge pull request #5825 from SomberNight/201912_local_tx_can_be_partial
wallet: allow saving partial tx as local (if it has a txid)
2019-12-15 16:40:46 +01:00
SomberNight
869a728317 wallet: use abstract base classes 2019-12-10 19:34:44 +01:00
SomberNight
30dcab0877 wallet: allow saving partial txns as local (but require txid) 2019-12-08 04:32:44 +01:00
SomberNight
8dbbc21aff wallet: better (outgoing) invoice "paid" detection
- no more passing around "invoice" in GUIs, invoice "paid" detection is now handled by wallet logic
- a tx can now pay for multiple invoices
- an invoice can now be paid by multiple txs (through partial payments)
- new data structure in storage: prevouts_by_scripthash
  - type: scripthash -> set of (outpoint, value)
  - also, storage upgrade to build this for existing wallets
2019-11-29 15:06:16 +01:00
ThomasV
61dfcba092 Refactor channel states:
- persisted states are saved
 - state transitions are checked
 - transient states are stored in channel.peer_state
 - new channel states: 'PREOPENING', 'FUNDED' and 'REDEEMED'
 - upgrade storage to version 21
2019-11-22 20:14:54 +01:00
SomberNight
e6c841d05f psbt: put fake xpubs into globals. keystores handle xfp/der_prefix missing 2019-11-04 22:24:59 +01:00
SomberNight
bafe8a2fff integrate PSBT support natively. WIP 2019-11-04 22:24:36 +01:00
ThomasV
7b828359c6 simplify get_tx_fee 2019-09-12 12:26:49 +02:00
SomberNight
482605edbb wallet: organise get_tx_fee. store calculated fees. storage version 19. 2019-09-12 08:59:27 +02:00
SomberNight
b138fff9a5 wallet: txi/txo small clean-up 2019-09-12 04:07:17 +02:00
SomberNight
8c91deb621 storage: better error msg on unsupported seed version 2019-08-09 18:11:50 +02:00
SomberNight
c6a54f05f5 wallet: some performance optimisations for get_receiving_addresses
jsondb takes a copy of the whole self.receiving_addresses
good for avoiding race conditions but horrible for performance...
this significantly speeds up at least
- synchronize_sequence, and
- is_beyond_limit (used by Qt AddressList)
2019-06-28 20:20:24 +02:00
SomberNight
e3c26d7c7a json_db: fix remove_spent_outpoint
method should make sure prevout_n is str...
also wrote failing test
2019-06-15 03:51:11 +02:00
SomberNight
6bdc6f559c storage: fix bug in convert_version_17
closes #5400
2019-06-06 19:51:37 +02:00
SomberNight
53d189fc7a storage: fix some madness about get_data_ref() and put() interacting badly
previously load_transactions() had to be called before upgrade();
now we reverse this order.

to reproduce/illustrate issue, before this commit:

try running convert_version_17 and convert_version_18
(e.g. see testcase test_upgrade_from_client_2_9_3_old_seeded_with_realistic_history)
and then in qt console:
>> wallet.storage.db.get_data_ref('spent_outpoints') == wallet.storage.db.spent_outpoints
False
>> wallet.storage.db.get_data_ref('verified_tx3') == wallet.storage.db.verified_tx
False
2019-06-06 19:49:06 +02:00
SomberNight
fd58a0cb20 json_db: enforce order of 'load_transactions' and 'upgrade'
fixes #5331
2019-05-13 19:21:26 +02:00
SomberNight
3385a94753 logging: basics 2019-05-02 15:19:03 +02:00
ThomasV
56ced1dfd5 let DB handle addresses 2019-03-05 08:20:34 +01:00
SomberNight
514d0ae958 wallet: towards restoring previous performance 2 2019-03-04 18:16:48 +01:00
SomberNight
b134f04fef storage: fix convert_version_18 2019-03-01 19:54:09 +01:00
SomberNight
9ecb504739 storage: fix convert_version_17 and add new test case
follow-up 121b8048b0
2019-03-01 19:46:23 +01:00
SomberNight
121b8048b0 json_db: store Transaction objects in memory, not raw hex
to avoid deserializing the same tx multiple times
2019-03-01 17:59:22 +01:00
SomberNight
2ad73050b3 wallet: towards restoring previous performance 2019-03-01 17:59:22 +01:00
ThomasV
2fd4cdcaa9 json_db: add missing lock 2019-02-28 16:11:19 +01:00
ThomasV
943d1ba8f2 json_db: private methods. return tx in remove_transaction 2019-02-28 16:03:36 +01:00
ThomasV
1e519f2dd0 json_db: make get operations threadsafe 2019-02-28 15:47:42 +01:00
ThomasV
d74f0c0947 storage_db: fix tests, add modified flag to db class 2019-02-28 12:09:36 +01:00
ThomasV
048eb01300 fix json_db list 2019-02-28 10:04:44 +01:00
ThomasV
cb9dcb8e26 load wallet plugin in Wallet factory, fix trustedcoin wizard on kivy 2019-02-28 09:02:58 +01:00
ThomasV
d73f7a2c10 jsondb: convert lists to sets 2019-02-28 09:02:58 +01:00
SomberNight
4b36114d0d small fixups 2019-02-28 09:02:58 +01:00
ThomasV
791e680a96 abstract database away from wallet and address_synchronizer 2019-02-28 09:02:58 +01:00
ThomasV
7f2083f667 separate storage and database (JsonDB) 2019-02-28 09:02:58 +01:00