Application logs are an essential piece of information that provides crucial insights into the inner workings of an application. This includes valuable information such as events, errors, and user interactions that would aid an application developer or an operations support engineer to debug and provide support. However, when these logs are presented in languages other than English, it creates a significant hurdle for developers who can’t read the content, and hinders the support team’s ability to identify and address issues promptly.
In this post, we explore a solution on how you can unlock language barriers using Amazon Translate, a fully managed neural machine translation service for translating text to and from English across a wide range of supported languages. The solution will complement your existing logging workflows by automatically translating all your applications logs in Amazon CloudWatch in real time, which can alleviate the challenges posed by non-English application logs.
This solution shows you how you can use three key services to automate the translation of your application logs in an event-driven manner:
The following diagram illustrates the solution architecture.
The workflow consists of the following steps:
The entire process happens automatically in real time, and your developers will be able to access the translated application logs from the CloudWatch log groups with no change in how your existing application writes logs to CloudWatch.
To follow through the instructions in this solution, you need an AWS account with an AWS Identity and Access Management (IAM) user who has permission to AWS CloudFormation, Amazon Translate, CloudWatch, Lambda, and IAM.
To get started, launch the following CloudFormation template to create a Lambda function, two CloudWatch log groups, and IAM role. Proceed to deploy with the default settings. This template takes about 1 minute to complete.
After the stack is created successfully, you can review the Lambda function by navigating to the Lambda console and locating the function translate-application-logs.
You can observe that there is a CloudWatch Logs trigger added to the function.
You can view the details of the trigger configuration by navigating to the Configuration tab and choosing Triggers in the navigation pane.
You can confirm that the trigger has been configured to subscribe to log events from the log group /applicationlogs. This is where your non-English application logs will be written to.
Next, choose Environment variables in the navigation pane.
Two environment variables are provided here:
For a list of supported languages, refer to Supported languages and language codes.
Next, go to the Code tab and review the function logic:
import json, boto3, gzip, base64, os translate = boto3.client(service_name=’translate’, region_name=os.environ[‘AWS_REGION’], use_ssl=True) logs = boto3.client(‘logs’) def lambda_handler(event, context): # retrieve log messages encoded_zipped_data = event[‘awslogs’][‘data’] zipped_data = base64.b64decode(encoded_zipped_data) data = gzip.decompress(zipped_data) json_log = json.loads(data) logGroup = json_log[‘logGroup’]+’-‘+os.environ[‘target_language’] logStream = json_log[‘logStream’] # check if log group exists, create if not dlg = logs.describe_log_groups(logGroupNamePrefix=logGroup) if len(dlg[‘logGroups’]) == 0: logs.create_log_group(logGroupName=logGroup) # check if log stream exists, create if not dls = logs.describe_log_streams(logGroupName=logGroup, logStreamNamePrefix=logStream) if len(dls[‘logStreams’]) == 0: logs.create_log_stream(logGroupName=logGroup, logStreamName=logStream) # translate log event messages from source language to target language for logevent in json_log[‘logEvents’]: logevent[‘message’] = translate.translate_text(Text=logevent[‘message’], SourceLanguageCode=os.environ[‘source_language’], TargetLanguageCode=os.environ[‘target_language’]).get(‘TranslatedText’) del logevent[‘id’] # write translated log events back to a different log group in CloudWatch logs.put_log_events( logGroupName = logGroup, logStreamName = logStream, logEvents = json_log[‘logEvents’] ) # return success return { ‘statusCode’: 200, ‘body’: ‘Translation success!’ }
Finally, to test the solution, you can create a log message through the CloudWatch console and choose the created log group and log stream.
After creating your log messages, you will be able to see it translated immediately.
To clean up the resources created in this post, delete the CloudFormation stack via the CloudFormation console.
This post addressed the challenge faced by developers and support teams when application logs are presented in languages other than English, making it difficult for them to debug and provide support. The proposed solution uses Amazon Translate to automatically translate non-English logs in CloudWatch, and provides step-by-step guidance on deploying the solution in your environment. Through this implementation, developers can now seamlessly bridge the language barrier, empowering them to address issues swiftly and effectively.
Try out this implementation and let us know your thoughts in the comments.
Xan Huang is a Senior Solutions Architect with AWS and is based in Singapore. He works with major financial institutions to design and build secure, scalable, and highly available solutions in the cloud. Outside of work, Xan spends most of his free time with his family and documenting his daughter’s growing up journey.