Multi-Document Transactions on MongoDB 4.0

What’s New on MongoDB 4.0

> s = db.getMongo().startSession()
session { "id" : UUID("3bf55e90-5e88-44aa-a59e-a30f777f1d89") }
> s.startTransaction()
> db.coll01.insert({x: 1, y: 1})
WriteResult({ "nInserted" : 1 })
> db.coll02.insert({x: 1, y: 1})
WriteResult({ "nInserted" : 1 })
> s.commitTransaction() (or s.abortTransaction() rollback transaction)

Python Version

with client.start_session() as s:
s.start_transaction()
collection_one.insert_one(doc_one, session=s)
collection_two.insert_one(doc_two, session=s)
s.commit_transaction()

Java Version

try (ClientSession clientSession = client.startSession()) {
clientSession.startTransaction();
collection.insertOne(clientSession, docOne);
collection.insertOne(clientSession, docTwo);
clientSession.commitTransaction();
}

Session

  1. lsid: The ID of the session where the request is located, also known as the logic session id.
  2. txnNmuber: The transaction number corresponding to the request, which must be monotonically incremented within a Session.
  3. stmtIds: It corresponds the ID to each operation in the request (in the case of insert, an insert command can insert multiple documents).
  1. causalConsistency: Whether to provide the semantics of causal consistency, if it is set to true, MongoDB guarantees the semantics of “read your own write” no matter which node it reads. See causal consistency
  2. writeConcern: MongoDB supports client-side flexible configuration write policies (writeConcern) to meet the needs of different scenarios.
  3. readConcern: MongoDB can customize the write strategy through writeConcern. After version 3.2, readConcern was introduced to flexibly customize the read strategy.
  4. readPreference: Rules for selecting nodes when setting read, see read preference
  5. retryWrites: If it is set to true, in the scenario of replica sets, MongoDB will automatically retry the scene where re-selection occurs; see retryable write

ACID

Atomic

Consistency

Isolation

Durability

Transaction and Replica

"ts" : Timestamp(1530696933, 1), "t" : NumberLong(1), "h" : NumberLong("4217817601701821530"), "v" : 2, "op" : "c", "ns" : "admin.$cmd", "wall" : ISODate("2018-07-04T09:35:33.549Z"), "lsid" : { "id" : UUID("e675c046-d70b-44c2-ad8d-3f34f2019a7e"), "uid" : BinData(0,"47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=") }, "txnNumber" : NumberLong(0), "stmtId" : 0, "prevOpTime" : { "ts" : Timestamp(0, 0), "t" : NumberLong(-1) }, "o" : { "applyOps" : [ { "op" : "i", "ns" : "test.coll2", "ui" : UUID("a49ccd80-6cfc-4896-9740-c5bff41e7cce"), "o" : { "_id" : ObjectId("5b3c94d4624d615ede6097ae"), "x" : 20000 } }, { "op" : "i", "ns" : "test.coll3", "ui" : UUID("31d7ae62-fe78-44f5-ba06-595ae3b871fc"), "o" : { "_id" : ObjectId("5b3c94d9624d615ede6097af"), "x" : 20000 } } ] } }
  1. Get the current batch (the back-end continues to pull the oplog into the batch)
  2. Set the OplogTruncateAfterPoint timestamp to the timestamp of the first oplog in the batch (stored in the set local.replset.oplogTruncateAfterPoint)
  3. Write all the oplogs in the batch to the set local.oplog.rs. If the number of oplogs is large, the write acceleration will be used.
  4. Clear OplogTruncateAfterPoint, and mark oplog to be completely successfully written. If a crash occurs before this step is completed, after restarting and recovery, it is found that oplogTruncateAfterPoint is set, then the oplog is truncated to the timestamp to restore a consistent status.
  5. The oplog is divided into multiple threads for concurrent replay. In order to improve the efficiency of the concurrency, the oplog generated by the transaction contains all the modifications, and will be divided into multiple threads according to the document ID like the oplog of a normal single operation.
  6. Update the ApplyThrough timestamp to the timestamp of the last oplog in the batch. After marking the next restart, resynchronize from that location. If it fails before this step, the oplog will be pulled from the last value of ApplyThrough (the last oplog of the previous batch).
  7. Update the oplog visible timestamp. If other nodes synchronize from the slave node, this part of the newly written oplog can be read.
  8. Update the local snapshot (timestamp) and the new write will be visible to users.

Transaction and Storage Engine

Unified Transaction Timing

/*
* __wt_txn_visible --
* Can the current transaction see the given ID / timestamp?
*/
static inline bool
__wt_txn_visible(
WT_SESSION_IMPL *session, uint64_t id, const wt_timestamp_t *timestamp)
{
if (!__txn_visible_id(session, id))
return (false);
/* Transactions read their writes, regardless of timestamps. */
if (F_ISSET(&session->txn, WT_TXN_HAS_ID) && id == session->txn.id)
return (true);
#ifdef HAVE_TIMESTAMPS
{
WT_TXN *txn = &session->txn;
/* Timestamp check. */
if (!F_ISSET(txn, WT_TXN_HAS_TS_READ) || timestamp == NULL)
return (true);
return (__wt_timestamp_cmp(timestamp, &txn->read_timestamp) <= 0);
}
#else
WT_UNUSED(timestamp);
return (true);
#endif
}

Impact of Transaction on the Cache

  1. The life cycle of a transaction cannot exceed transactionLifetimeLimitSeconds (60 seconds by default), which can be modified online.
  2. The number of documents modified by a transaction cannot exceed 1,000, which cannot be modified.
  3. The oplog generated by a transaction modification cannot exceed 16 MB, which is the document size limit of MongoDB. The oplog is also a normal document and must comply with this constraint.

Read as of a Timestamp and Oldest Timestamp

Engine Layer Rollback and Stable Timestamp

Distributed Transactions

ApsaraDB for MongoDB

--

--

--

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Agile Fundamentals Overview.

MaxCompute and DataWorks Security Management Guide: Examples

The Perfect Language is a Combination of Java and Python

Alibaba Cloud RPA: Compiling an RPA Application in Visual Mode (Part 7)

Up and Running with Alibaba Cloud Container Registry

How to Integrate Alibaba Cloud WAF Log with Splunk

Flutter Analysis and Practice: AOP Design Practices

Serialization Implementation Method Based on Generic Programming

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Alibaba Cloud

Alibaba Cloud

Follow me to keep abreast with the latest technology news, industry insights, and developer trends. Alibaba Cloud website:https://www.alibabacloud.com

More from Medium

OAuth2.0 and OpenID

Asynchronous workflows powered by AWS-Lambda

Elasticsearch — How to Optimise Index and Search Performance

Getting started with Apache APISIX