Home  ›  How to Ensure Cloudwatch Logs Finish Uploading Before Shutting Down

How to Ensure Cloudwatch Logs Finish Uploading Before Shutting Down

Written By Wednesday, April 13, 2022 Add Comment Edit

Amazon CloudWatch Logs

  1. Concluding updated
  2. Save as PDF

Nosotros recommend collecting Amazon CloudWatch Logs using our AWS Lambda function to subscribe to your CloudWatch Log Group. Our AWS Lambda function converts the CloudWatch log format into a format that is compatible with Sumo, then POSTs the data directly to a Sumo HTTP Source.  This is the preferred method for the following types of information that are delivered through Amazon CloudWatch Logs:

  • Custom CloudWatch log data. The AWS Lambda function should handle whatsoever log data. Nevertheless, you should make sure to test this with your actual data, to ensure that unusually formatted logs are parsed correctly.
  • Amazon VPC Flow Logs. The AWS Lambda role is compatible with the Sumo Amazon VPC Flow Logs App.
  • AWS Lambda logs.The AWS Lambda role is built for logs generated by your AWS Lambda functions and is compatible with our Sumo AWS Lambda App.

Collect CloudWatch Logs using a CloudFormation Template

This page has instructions for creating AWS resources using a Sumo-provided CloudFormation template. The template specifies the resources necessary to send Amazon CloudWatch Logs to Sumo, including a Lambda function for sending logs, another Lambda role configured with a dead letter of the alphabet queue for resending letters as necessary, and associated roles and permissions. For more than data about the resources created, encounter Download the CloudFormation template.

CloudFormation based approach is recommended. If you lot would rather manually configure a Lambda function see Collect Amazon CloudWatch Logs with Lambda Role.

Download the CloudFormation template

If yous want to make any of the optional modifications described in this department, download the DLQLambdaCloudFormation.json CloudFormation template from https://s3.amazonaws.com/appdev-cloudformation-templates/DLQLambdaCloudFormation.json. Otherwise, proceed to Create a stack on the AWS CloudFormation console.

When y'all upload the template to AWS, it creates the AWS resources described in the tabular array below.

Resource Proper name Description
SumoCWLogGroup A log group that has a subscription filter (SumoCWLogSubsriptionFilter)  associated with it that delivers real time logs to Sumo's CloudWatch Lambda function (SumoCWLogsLambda).
SumoCWLogsLambda A Lambda function responsible for sending information to the Sumo HTTP Source URL. It is configured with a expressionless alphabetic character queue (SumoCWDeadLetterQueue) that receives letters that can't be processed successfully. You lot can subscribe other logs to this office except its own log grouping.
SumoCWProcessDLQLambda A Lambda function responsible for reading messages from the expressionless alphabetic character queue and resending letters. This function is periodically triggered past AWS CloudWatch Events using a schedule dominion (SumoCWProcessDLQScheduleRule).
SumoCWLambdaPermission Permission to CloudWatch Logs for invoking Lambda functions.
SumoCWLambdaExecutionRole IAM Role for the two Lambda functions. The function includes policies for creating CloudWatch Logs, running CRUD operations on the expressionless letter queue  (SumoCWDeadLetterQueue), and invoking Lambda functions.
SumoCWEventsInvokeLambdaPermission Permission for CloudWatch events to trigger the SumoCWProcessDLQLambda Lambda function.
SumoCWSpilloverAlarm An alarm that is triggered if the number of letters in the Dead Letter Queue exceeds the threshold divers in the CoudFormation template (default is 100000). The alert is configured with a "send email" action (SumoCWEmailSNSTopic). You must verify receipt of emails sent to the electronic mail endpoint defined in CloudFormation template.

Tailor the CloudFormation template

Before you upload the CloudFormation template, there are some optional configuration steps.

If you want to use the CloudFormation Template every bit is, proceed to Create a stack on the AWS CloudFormation console.

Configure surroundings variables for Lambda functions

The following AWS Lambda surround variables are supported in both the Lambda functions. Both the functions should have same environment variables values configured to avoid inconsistencies.

Environs Variable Description
SOURCE_CATEGORY_OVERRIDE (Optional) You can use this variable to override the _sourceCategory configured for the HTTP Source.
SOURCE_FIELDS_OVERRIDE (Optional) You can use this variable to override the custom metadata fields configured for the HTTP Source. Example: key1=value1,key2=value2
SOURCE_HOST_OVERRIDE (Optional) You lot can use this variable to override the_sourceHost configured for the HTTP Source.
SOURCE_NAME_OVERRIDE (Optional) You can employ this variable to override the_sourceName configured for the HTTP Source.

If y'all are configuring log collection for VPC menstruation logs, see the Surroundings variables for VPC menstruation log drove department on Collect Amazon VPC Menstruum Logs from CloudWatch using CloudFormation.

Define variables in the Environmentsection of the Cloud Formation template.

"Environment": {                  "Variables": {                      "SOURCE_CATEGORY_OVERRIDE": "<insert-value-here>"                      "SOURCE_HOST_OVERRIDE": "<insert-value-here>"                      "SOURCE_NAME_OVERRIDE": "<insert-value-here>"                      "SOURCE_FIELDS_OVERRIDE": "<insert-value-here>"                     }                 }
Configure threshold for DeadLetterQueue

In the CloudFormation template, define the number of letters in the Dead Letter Queue that will trigger the SumoCWSpilloverAlarm, using the Threshold attribute in the alarm definition.

"SumoCWSpilloverAlarm":{             "Blazon":"AWS::CloudWatch::Warning",             "Backdrop":{                 "AlarmActions":[                     {                         "Ref":"SumoCWEmailSNSTopic"                     }                 ],                 "AlarmDescription":"Notify via email if number of messages in DeadLetterQueue exceeds threshold",                 "ComparisonOperator":"GreaterThanThreshold",                 "Dimensions":[                   {                     "Name": "QueueName",                     "Value": "SumoCWDeadLetterQueue"                   }                 ],                 "EvaluationPeriods":"1",                 "MetricName":"ApproximateNumberOfMessagesVisible",                 "Namespace":"AWS/SQS",                 "Menses":"3600",                 "Statistic":"Sum",                 "Threshold":"100000"             },             "DependsOn": ["SumoCWEmailSNSTopic"]
Remove alarm resources

(Optional) If you do non want the SumoCWSpilloverAlarm alarm to be created, remove the definitions of the SumoCWEmailSNSTopic and SumoCWSpilloverAlarm resources from the CloudFormation template. Delete the sections shown below.

"SumoCWEmailSNSTopic": {             "Blazon":"AWS::SNS::Topic",             "Backdrop":{                 "Subscription":[ {                     "Endpoint" : "hpal@sumologic.com",                     "Protocol" : "email"                 }]             }         },         "SumoCWSpilloverAlarm":{             "Type":"AWS::CloudWatch::Warning",             "Properties":{                 "AlarmActions":[                     {                         "Ref":"SumoCWEmailSNSTopic"                     }                 ],                 "AlarmDescription":"Notify via e-mail if number of messages in DeadLetterQueue exceeds threshold",                 "ComparisonOperator":"GreaterThanThreshold",                 "Dimensions":[                   {                     "Name": "QueueName",                     "Value": "SumoCWDeadLetterQueue"                   }                 ],                 "EvaluationPeriods":"1",                 "MetricName":"ApproximateNumberOfMessagesVisible",                 "Namespace":"AWS/SQS",                 "Period":"3600",                 "Statistic":"Sum",                 "Threshold":"100000"             },             "DependsOn": ["SumoCWEmailSNSTopic"]         }

Create a stack on the AWS CloudFormation panel

  1. Log in to the AWS Management Panel.
  2. Nether Management Tools, select CloudFormation.
  3. Create a new stack past clicking Create Stack, then select "With new resource (standard)."

CF_animated.gif

  1. On the Specify Template window, do one of the following:

  • If you accept downloaded and optionally modified the CloudFormation template, choose to Upload a template file, upload the DLQLambdaCloudFormation.json file, and and so click Next.

Create_Stack_dialog.png

  • Otherwise, if you did not modify the CloudFormation template, select Specify an Amazon S3 template URL and enter:
    https://s3.amazonaws.com/appdev-cloudformation-templates/DLQLambdaCloudFormation.json


Create_Stack_dialog_2.png

  1. Select Side by side and the Specify Stack Details window appears. Enter the following:
    • Stack Name.
    • EmailID (Optional) Used for alerts.
    • IncludeLogGroupInfo.  Set to true to include loggroup/logstream values in logs. The default value is Simulated. For AWS Lambda Logs IncludeLogGroupInfo must be prepare to True; for VPC Flow Logs it'south optional.
    • LogFormat. For VPC logs, choose either VPC-JSON (JSON format) or VPC-RAW (raw messages). The default value is Others.
    • LogStreamPrefix (Optional) Enter comma separated list of logStream name prefixes to filter by logStream. Please note this is seperate from a logGroup. This is used to but send certain logStreams within a cloudwatch logGroup(s).  LogGroup(s) still need to be subscribed to the created Lambda funciton (SumoCWLogsLambda-<Auto-Genereted-Suffix>), regardless of what is input for this value.
    • NumOfWorkers. (Optional) Increment this value to speed upwards dead letter queue (DLQ) processing.
    • SumoEndPointURL (Required). Enter the HTTP Source Accost URL from Add a Hosted Collector and HTTP Source.

Specify_Stack_Details.png

  1. Click Next. The Configure Stack Options screen will announced. You tin optinally add AWS tags to tag the resource created by this Cloudformation stack. Click Side by side to get to the final Review window.
  2. In the Review window, click the checkbox acknowledging that you sympathize the the template creates IAM resource, and click Create.

After few minutes you will see CREATE_COMPLETE in the Condition column.

Validate email accost for alarms

Log in to the email account whose address yous provided when performing the configuration described in Create a stack on the AWS CloudFormation console higher up. Await for an e-mail with bailiwick "AWS Notification - Subscription Confirmation", similar the example shown below.

aws-notification.png

To validate the email address, click Confirm subscription in the e-mail.

Dealing with alarms

If you receive an alarm email like the one shown in the previous section, the number of messages in the dead letter queue exceeds the threshold defined in the CloudFormation template, which past default is 100,000. This could be because:

  • SumoCWProcessDLQLambda may not be able to procedure messages as rapidly as the letters are received. In this case, you may want to use the Lambda console to increase the number of workers specified past the NUM_OF_WORKERS environment variable.

  • SumoCWProcessDLQLambda may exist unable process incoming messages because of an fault in the bulletin format or a configuration problem, for instance an mistake in the HTTP endpoint configuration. Test the role with the message in the Lambda panel to see whether it is able to process the message and transport it to Sumo.

Subscribe SumoCWLogsLambda to CloudWatch Log Groups

The process described to a higher place subscribes a unmarried Log Grouping, SumoCWLogGroup, to the SumoCWLogsLambda function. If y'all would similar to subscribe additional CloudWatch Log Groups to the SumoCWLogsLambda function, follow the instructions in the sections below.

Manually subscribe SumoCWLogsLambda to an existing CloudWatch Log Group

If you only demand to collect logs from a few boosted CloudWatch Log groups, yous can manually subscribe the SumoCWLogsLambda function to an existing CloudWatch Log Group using the instructions below.

  1. Log in to the AWS Management Console.
  2. Under Management Tools, select CloudWatch, then click Logs in the left- paw navigation carte.
  3. Select the radio button next to the CloudWatch Log Grouping that you desire to stream to Sumo Logic, click Actions, then click Stream to AWS Lambda.
    stream-to-aws-lambda.png
  4. Select the Lambda function that begins with "SumoCWLogsLambda", and then click Adjacent.
    lambda-function.png
  5. Select the appropriate log format, then click Next.
  6. Confirm the details on the next screen, then click Start Streaming.

Auto-subscribe other log groups to SumoCWLogsLambda part

 If you want to collect logs from multiple Log Groups, you tin can employ Sumo'southward LogGroup Lambda Connector to subscribe additional Log Groups to the Lambda function. To practice so, follow the instructions in Car-Subscribe AWS Log Groups to a Lambda Role.  When you lot edit the CloudFormation template for the connector, point the LAMBDA_ARN environment variable to the SumoCWLogsLambda function.

Alternate drove methods

If yous tin can't use AWS Lambda or CloudFormation to collect logs from CloudWatch, choose i of the following methods:

  • A Lambda office without CloudFormation. To manually configure a Lambda function, see Collect Amazon CloudWatch Logs with Lambda Office.
  • Amazon Kinesis. If AWS Lambda is not bachelor to you lot, or you demand increased delivery reliability, review how to add Amazon Kinesis to the integration as described in AWS Kinesis Firehose for Logs Source.
  • The Sumo Logic Collector and a Script. If y'all have a relatively small amount of CloudWatch logs to collect, and you practice not want to set up any additional AWS infrastructure, you lot tin install the Sumo Logic Collector amanuensis locally, and run a script that we have developed for CloudWatch logs, with a special focus on Amazon VPC Catamenia Logs.  Encounter the instructions for collecting CloudWatch logs using a collector script.

grandefeeirid.blogspot.com

Source: https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Amazon-CloudWatch-Logs

0 Response to "How to Ensure Cloudwatch Logs Finish Uploading Before Shutting Down"

Post a Comment

Subscribe to: Post Comments (Atom)

Iklan Atas Artikel

Iklan Tengah Artikel 1

Iklan Tengah Artikel 2

Iklan Bawah Artikel