BatchWriteItem() public method

The BatchWriteItem operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 put or delete requests. Individual items to be written can be as large as 400 KB.

BatchWriteItem cannot update items. To update items, use the UpdateItem API.

The individual PutItem and DeleteItem operations specified in BatchWriteItem are atomic; however BatchWriteItem as a whole is not. If any requested operations fail because the table's provisioned throughput is exceeded or an internal processing failure occurs, the failed operations are returned in the UnprocessedItems response parameter. You can investigate and optionally resend the requests. Typically, you would call BatchWriteItem in a loop. Each iteration would check for unprocessed items and submit a new BatchWriteItem request with those unprocessed items until all items have been processed.

Note that if none of the items can be processed due to insufficient provisioned throughput on all of the tables in the request, then BatchWriteItem will return a ProvisionedThroughputExceededException.

If DynamoDB returns any unprocessed items, you should retry the batch operation on those items. However, we strongly recommend that you use an exponential backoff algorithm. If you retry the batch operation immediately, the underlying read or write requests can still fail due to throttling on the individual tables. If you delay the batch operation using exponential backoff, the individual requests in the batch are much more likely to succeed.

For more information, see Batch Operations and Error Handling in the Amazon DynamoDB Developer Guide.

With BatchWriteItem, you can efficiently write or delete large amounts of data, such as from Amazon Elastic MapReduce (EMR), or copy data from another database into DynamoDB. In order to improve performance with these large-scale operations, BatchWriteItem does not behave in the same way as individual PutItem and DeleteItem calls would. For example, you cannot specify conditions on individual put and delete requests, and BatchWriteItem does not return deleted items in the response.

If you use a programming language that supports concurrency, you can use threads to write items in parallel. Your application must include the necessary logic to manage the threads. With languages that don't support threading, you must update or delete the specified items one at a time. In both situations, BatchWriteItem provides an alternative where the API performs the specified put and delete operations in parallel, giving you the power of the thread pool approach without having to introduce complexity into your application.

Parallel processing reduces latency, but each specified put and delete request consumes the same number of write capacity units whether it is processed in parallel or not. Delete operations on nonexistent items consume one write capacity unit.

If one or more of the following is true, DynamoDB rejects the entire batch write operation:

  • One or more tables specified in the BatchWriteItem request does not exist.

  • Primary key attributes specified on an item in the request do not match those in the corresponding table's primary key schema.

  • You try to perform multiple operations on the same item in the same BatchWriteItem request. For example, you cannot put and delete the same item in the same BatchWriteItem request.

  • There are more than 25 requests in the batch.

  • Any individual item in a batch exceeds 400 KB.

  • The total request size exceeds 16 MB.

/// An error occurred on the server side. /// /// An item collection is too large. This exception is only returned for tables that have /// one or more local secondary indexes. /// /// Your request rate is too high. The AWS SDKs for DynamoDB automatically retry requests /// that receive this exception. Your request is eventually successful, unless your retry /// queue is too large to finish. Reduce the frequency of requests and use exponential /// backoff. For more information, go to Error /// Retries and Exponential Backoff in the Amazon DynamoDB Developer Guide. /// /// The operation tried to access a nonexistent table or index. The resource might not /// be specified correctly, or its status might not be ACTIVE. ///
public BatchWriteItem ( BatchWriteItemRequest request ) : BatchWriteItemResponse
request Amazon.DynamoDBv2.Model.BatchWriteItemRequest Container for the necessary parameters to execute the BatchWriteItem service method.
return Amazon.DynamoDBv2.Model.BatchWriteItemResponse
コード例 #1
0
        public void BatchSamples()
        {
            EnsureTables();

            {
                #region BatchGet Sample 1

                // Define attributes to get and keys to retrieve
                List<string> attributesToGet = new List<string> { "Author", "Title", "Year" };
                List<Dictionary<string, AttributeValue>> sampleTableKeys = new List<Dictionary<string, AttributeValue>>
                {
                    new Dictionary<string, AttributeValue>
                    {
                        { "Author", new AttributeValue { S = "Mark Twain" } },
                        { "Title", new AttributeValue { S = "The Adventures of Tom Sawyer" } }
                    },
                    new Dictionary<string, AttributeValue>
                    {
                        { "Author", new AttributeValue { S = "Mark Twain" } },
                        { "Title", new AttributeValue { S = "Adventures of Huckleberry Finn" } }
                    }
                };

                // Construct get-request for first table
                KeysAndAttributes sampleTableItems = new KeysAndAttributes
                {
                    AttributesToGet = attributesToGet,
                    Keys = sampleTableKeys
                };

                #endregion

                #region BatchGet Sample 2

                // Define keys to retrieve
                List<Dictionary<string, AttributeValue>> authorsTableKeys = new List<Dictionary<string, AttributeValue>>
                {
                    new Dictionary<string, AttributeValue>
                    {
                        { "Author", new AttributeValue { S = "Mark Twain" } },
                    },
                    new Dictionary<string, AttributeValue>
                    {
                        { "Author", new AttributeValue { S = "Booker Taliaferro Washington" } },
                    }
                };

                // Construct get-request for second table
                //  Skip setting AttributesToGet property to retrieve all attributes
                KeysAndAttributes authorsTableItems = new KeysAndAttributes
                {
                    Keys = authorsTableKeys,
                };

                #endregion

                #region BatchGet Sample 3

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Construct table-keys mapping
                Dictionary<string, KeysAndAttributes> requestItems = new Dictionary<string, KeysAndAttributes>();
                requestItems["SampleTable"] = sampleTableItems;
                requestItems["AuthorsTable"] = authorsTableItems;

                // Construct request
                BatchGetItemRequest request = new BatchGetItemRequest
                {
                    RequestItems = requestItems
                };

                BatchGetItemResult result;
                do
                {
                    // Issue request and retrieve items
                    result = client.BatchGetItem(request);

                    // Iterate through responses
                    Dictionary<string, List<Dictionary<string, AttributeValue>>> responses = result.Responses;
                    foreach (string tableName in responses.Keys)
                    {
                        // Get items for each table
                        List<Dictionary<string, AttributeValue>> tableItems = responses[tableName];

                        // View items
                        foreach (Dictionary<string, AttributeValue> item in tableItems)
                        {
                            Console.WriteLine("Item:");
                            foreach (var keyValuePair in item)
                            {
                                Console.WriteLine("{0} : S={1}, N={2}, SS=[{3}], NS=[{4}]",
                                    keyValuePair.Key,
                                    keyValuePair.Value.S,
                                    keyValuePair.Value.N,
                                    string.Join(", ", keyValuePair.Value.SS ?? new List<string>()),
                                    string.Join(", ", keyValuePair.Value.NS ?? new List<string>()));
                            }
                        }
                    }

                    // Some items may not have been retrieved!
                    //  Set RequestItems to the result's UnprocessedKeys and reissue request
                    request.RequestItems = result.UnprocessedKeys;

                } while (result.UnprocessedKeys.Count > 0);

                #endregion
            }


            {
                #region BatchWrite Sample 1

                // Create items to put into first table
                Dictionary<string, AttributeValue> item1 = new Dictionary<string, AttributeValue>();
                item1["Author"] = new AttributeValue { S = "Mark Twain" };
                item1["Title"] = new AttributeValue { S = "A Connecticut Yankee in King Arthur's Court" };
                item1["Pages"] = new AttributeValue { N = "575" };
                Dictionary<string, AttributeValue> item2 = new Dictionary<string, AttributeValue>();
                item2["Author"] = new AttributeValue { S = "Booker Taliaferro Washington" };
                item2["Title"] = new AttributeValue { S = "My Larger Education" };
                item2["Pages"] = new AttributeValue { N = "313" };
                item2["Year"] = new AttributeValue { N = "1911" };

                // Create key for item to delete from first table
                //  Hash-key of the target item is string value "Mark Twain"
                //  Range-key of the target item is string value "Tom Sawyer, Detective"
                Dictionary<string, AttributeValue> keyToDelete1 = new Dictionary<string, AttributeValue>
                {
                    { "Author", new AttributeValue { S = "Mark Twain" } },
                    { "Title", new AttributeValue { S = "Tom Sawyer, Detective" } }
                };

                // Construct write-request for first table
                List<WriteRequest> sampleTableItems = new List<WriteRequest>();
                sampleTableItems.Add(new WriteRequest
                {
                    PutRequest = new PutRequest { Item = item1 }
                });
                sampleTableItems.Add(new WriteRequest
                {
                    PutRequest = new PutRequest { Item = item2 }
                });
                sampleTableItems.Add(new WriteRequest
                {
                    DeleteRequest = new DeleteRequest { Key = keyToDelete1 }
                });

                #endregion

                #region BatchWrite Sample 2

                // Create key for item to delete from second table
                //  Hash-key of the target item is string value "Francis Scott Key Fitzgerald"
                Dictionary<string, AttributeValue> keyToDelete2 = new Dictionary<string, AttributeValue>
                {
                    { "Author", new AttributeValue { S = "Francis Scott Key Fitzgerald" } },
                };

                // Construct write-request for first table
                List<WriteRequest> authorsTableItems = new List<WriteRequest>();
                authorsTableItems.Add(new WriteRequest
                {
                    DeleteRequest = new DeleteRequest { Key = keyToDelete2 }
                });

                #endregion

                #region BatchWrite Sample 3

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Construct table-keys mapping
                Dictionary<string, List<WriteRequest>> requestItems = new Dictionary<string, List<WriteRequest>>();
                requestItems["SampleTable"] = sampleTableItems;
                requestItems["AuthorsTable"] = authorsTableItems;

                BatchWriteItemRequest request = new BatchWriteItemRequest { RequestItems = requestItems };
                BatchWriteItemResult result;
                do
                {
                    // Issue request and retrieve items
                    result = client.BatchWriteItem(request);

                    // Some items may not have been processed!
                    //  Set RequestItems to the result's UnprocessedItems and reissue request
                    request.RequestItems = result.UnprocessedItems;

                } while (result.UnprocessedItems.Count > 0);

                #endregion
            }
        }
コード例 #2
0
        public void SearchSamples()
        {
            RemoveTables();
            CreateLSITable();
            TableUtils.WaitUntilTableActive("SampleTable", TestClient);

            {
                // Create items to put into first table
                Dictionary<string, AttributeValue> item1 = new Dictionary<string, AttributeValue>();
                item1["Author"] = new AttributeValue { S = "Mark Twain" };
                item1["Title"] = new AttributeValue { S = "A Connecticut Yankee in King Arthur's Court" };
                item1["Pages"] = new AttributeValue { N = "575" };
                Dictionary<string, AttributeValue> item2 = new Dictionary<string, AttributeValue>();
                item2["Author"] = new AttributeValue { S = "Booker Taliaferro Washington" };
                item2["Title"] = new AttributeValue { S = "My Larger Education" };
                item2["Pages"] = new AttributeValue { N = "313" };
                item2["Year"] = new AttributeValue { N = "1911" };

                // Construct write-request for first table
                List<WriteRequest> sampleTableItems = new List<WriteRequest>();
                sampleTableItems.Add(new WriteRequest
                {
                    PutRequest = new PutRequest { Item = item1 }
                });
                sampleTableItems.Add(new WriteRequest
                {
                    PutRequest = new PutRequest { Item = item2 }
                });
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();
                client.BatchWriteItem(new BatchWriteItemRequest
                {
                    RequestItems = new Dictionary<string, List<WriteRequest>>
                    {
                        { "SampleTable", sampleTableItems }
                    }
                });

                PutSample();
            }


            {
                #region Query Sample

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Define item hash-key to be string value "Mark Twain"
                AttributeValue hashKey = new AttributeValue { S = "Mark Twain" };

                // Define query condition to search for range-keys that begin with the string "The Adventures"
                Condition condition = new Condition
                {
                    ComparisonOperator = "BEGINS_WITH",
                    AttributeValueList = new List<AttributeValue>
                    {
                        new AttributeValue { S = "The Adventures" }
                    }
                };

                // Create the key conditions from hashKey and condition
                Dictionary<string, Condition> keyConditions = new Dictionary<string, Condition>
                {
                    // Hash key condition. ComparisonOperator must be "EQ".
                    { 
                        "Author",
                        new Condition
                        {
                            ComparisonOperator = "EQ",
                            AttributeValueList = new List<AttributeValue> { hashKey }
                        }
                    },
                    // Range key condition
                    {
                        "Title",
                        condition
                    }
                };

                // Define marker variable
                Dictionary<string, AttributeValue> startKey = null;

                do
                {
                    // Create Query request
                    QueryRequest request = new QueryRequest
                    {
                        TableName = "SampleTable",
                        ExclusiveStartKey = startKey,
                        KeyConditions = keyConditions
                    };

                    // Issue request
                    var result = client.Query(request);

                    // View all returned items
                    List<Dictionary<string, AttributeValue>> items = result.Items;
                    foreach (Dictionary<string, AttributeValue> item in items)
                    {
                        Console.WriteLine("Item:");
                        foreach (var keyValuePair in item)
                        {
                            Console.WriteLine("{0} : S={1}, N={2}, SS=[{3}], NS=[{4}]",
                                keyValuePair.Key,
                                keyValuePair.Value.S,
                                keyValuePair.Value.N,
                                string.Join(", ", keyValuePair.Value.SS ?? new List<string>()),
                                string.Join(", ", keyValuePair.Value.NS ?? new List<string>()));
                        }
                    }

                    // Set marker variable
                    startKey = result.LastEvaluatedKey;
                } while (startKey != null && startKey.Count > 0);

                #endregion
            }

            {
                #region Query Local Secondary Index Sample

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Define item hash-key to be string value "Mark Twain"
                AttributeValue hashKey = new AttributeValue { S = "Mark Twain" };

                // Define query condition to search for range-keys ("Year", in "YearsIndex") that are less than 1900
                Condition condition = new Condition
                {
                    ComparisonOperator = "LT",
                    AttributeValueList = new List<AttributeValue>
                    {
                        new AttributeValue { N = "1900" }
                    }
                };

                // Create the key conditions from hashKey and condition
                Dictionary<string, Condition> keyConditions = new Dictionary<string, Condition>
                {
                    // Hash key condition. ComparisonOperator must be "EQ".
                    { 
                        "Author",
                        new Condition
                        {
                            ComparisonOperator = "EQ",
                            AttributeValueList = new List<AttributeValue> { hashKey }
                        }
                    },
                    // Range key condition
                    {
                        "Year", // Reference the correct range key when using indexes
                        condition
                    }
                };

                // Define marker variable
                Dictionary<string, AttributeValue> startKey = null;

                do
                {
                    // Create Query request
                    QueryRequest request = new QueryRequest
                    {
                        TableName = "SampleTable",
                        ExclusiveStartKey = startKey,
                        KeyConditions = keyConditions,
                        IndexName = "YearsIndex" // Specify the index to query against
                    };

                    // Issue request
                    var result = client.Query(request);

                    // View all returned items
                    List<Dictionary<string, AttributeValue>> items = result.Items;
                    foreach (Dictionary<string, AttributeValue> item in items)
                    {
                        Console.WriteLine("Item:");
                        foreach (var keyValuePair in item)
                        {
                            Console.WriteLine("{0} : S={1}, N={2}, SS=[{3}], NS=[{4}]",
                                keyValuePair.Key,
                                keyValuePair.Value.S,
                                keyValuePair.Value.N,
                                string.Join(", ", keyValuePair.Value.SS ?? new List<string>()),
                                string.Join(", ", keyValuePair.Value.NS ?? new List<string>()));
                        }
                    }

                    // Set marker variable
                    startKey = result.LastEvaluatedKey;
                } while (startKey != null && startKey.Count > 0);

                #endregion
            }

            {
                #region Scan Sample

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Define scan conditions
                Dictionary<string, Condition> conditions = new Dictionary<string, Condition>();

                // Title attribute should contain the string "Adventures"
                Condition titleCondition = new Condition();
                titleCondition.ComparisonOperator = ComparisonOperator.CONTAINS;
                titleCondition.AttributeValueList.Add(new AttributeValue { S = "Adventures" });
                conditions["Title"] = titleCondition;

                // Pages attributes must be greater-than the numeric value "200"
                Condition pagesCondition = new Condition();
                pagesCondition.ComparisonOperator = ComparisonOperator.GT;;
                pagesCondition.AttributeValueList.Add(new AttributeValue { N = "200" });
                conditions["Pages"] = pagesCondition;


                // Define marker variable
                Dictionary<string, AttributeValue> startKey = null;

                do
                {
                    // Create Scan request
                    ScanRequest request = new ScanRequest
                    {
                        TableName = "SampleTable",
                        ExclusiveStartKey = startKey,
                        ScanFilter = conditions
                    };

                    // Issue request
                    ScanResult result = client.Scan(request);

                    // View all returned items
                    List<Dictionary<string, AttributeValue>> items = result.Items;
                    foreach (Dictionary<string, AttributeValue> item in items)
                    {
                        Console.WriteLine("Item:");
                        foreach (var keyValuePair in item)
                        {
                            Console.WriteLine("{0} : S={1}, N={2}, SS=[{3}], NS=[{4}]",
                                keyValuePair.Key,
                                keyValuePair.Value.S,
                                keyValuePair.Value.N,
                                string.Join(", ", keyValuePair.Value.SS ?? new List<string>()),
                                string.Join(", ", keyValuePair.Value.NS ?? new List<string>()));
                        }
                    }

                    // Set marker variable
                    startKey = result.LastEvaluatedKey;
                } while (startKey != null && startKey.Count > 0);

                #endregion
            }

            {
                // Create lots of items to put into first table
                var table = Amazon.DynamoDBv2.DocumentModel.Table.LoadTable(TestClient, "SampleTable");
                var batchWrite = table.CreateBatchWrite();
                for (int i = 0; i < 100; i++)
                {
                    var document = new Amazon.DynamoDBv2.DocumentModel.Document();
                    document["Author"] = "FakeAuthor" + i;
                    document["Title"] = "Book" + i;
                    document["Pages"] = (180 + i);
                    document["Year"] = 1900 + i;
                    batchWrite.AddDocumentToPut(document);
                }
                batchWrite.Execute();
            }


            {
                #region Parallel Scan Sample

                // Create a client
                AmazonDynamoDBClient client = new AmazonDynamoDBClient();

                // Define scan conditions
                Dictionary<string, Condition> conditions = new Dictionary<string, Condition>();

                // Pages attributes must be greater-than the numeric value "200"
                Condition pagesCondition = new Condition();
                pagesCondition.ComparisonOperator = ComparisonOperator.GT;
                pagesCondition.AttributeValueList.Add(new AttributeValue { N = "200" });
                conditions["Pages"] = pagesCondition;

                // Setup 10 simultaneous threads, each thread calling Scan operation
                // with its own segment value.
                int totalSegments = 10;
                Parallel.For(0, totalSegments, segment =>
                {
                    // Define marker variable
                    Dictionary<string, AttributeValue> startKey = null;

                    do
                    {
                        // Create Scan request
                        ScanRequest request = new ScanRequest
                        {
                            TableName = "SampleTable",
                            ExclusiveStartKey = startKey,
                            ScanFilter = conditions,
                            // Total segments to split the table into
                            TotalSegments = totalSegments,
                            // Current segment to scan
                            Segment = segment
                        };

                        // Issue request
                        var result = client.Scan(request);

                        // Write returned items to file
                        string path = string.Format("ParallelScan-{0}-of-{1}.txt", totalSegments, segment);
                        List<Dictionary<string, AttributeValue>> items = result.Items;
                        using (Stream stream = File.OpenWrite(path))
                        using (StreamWriter writer = new StreamWriter(stream))
                        {
                            foreach (Dictionary<string, AttributeValue> item in items)
                            {
                                writer.WriteLine("Item:");
                                foreach (var keyValuePair in item)
                                {
                                    writer.WriteLine("{0} : S={1}, N={2}, SS=[{3}], NS=[{4}]",
                                        keyValuePair.Key,
                                        keyValuePair.Value.S,
                                        keyValuePair.Value.N,
                                        string.Join(", ", keyValuePair.Value.SS ?? new List<string>()),
                                        string.Join(", ", keyValuePair.Value.NS ?? new List<string>()));
                                }
                            }
                        }

                        // Set marker variable
                        startKey = result.LastEvaluatedKey;
                    } while (startKey != null && startKey.Count > 0);
                });

                #endregion
            }

        }