Stay up to date
We'll highlight new content in your collectives with a blue activity indicator on navigation items and posts.
Manage preferences
Questions
Browse questions with relevant AWS tags
47,889 questions
No answer
Advice
0
votes
0
replies
26
views
How do I add a static secret to AW1?
How do I add a static secret (which I now know is different from a generated secret) to AW1? I was provided this link to help me: https://git.taservs.net/ops/als-infra-aw1/tree/main/src/layer3/aw1-...
0
votes
0
answers
38
views
How get data from Amazon WorkDocs into the AWS S3 bucket?
I need to automate the process of moving files from Amazon WorkDocs to an S3 bucket.
Here’s what I want to achieve:
Periodically fetch files from a specific WorkDocs folder.
Upload or sync them to ...
0
votes
0
answers
60
views
Awswrangler: Parquet read into multiple of expected space
In a Lambda, I'm using AWS Wrangler to read data out of a date partitioned set of parquets and concatenate them together. I am doing this by calling wr.s3.read_parquet in a loop, compiling the loaded ...
- 70
0
votes
0
answers
34
views
Mount AWS EFS to ECS Fargate
I am mounting EFS to Fargate following the guide from here: https://repost.aws/knowledge-center/ecs-fargate-mount-efs-containers-tasks but receiving the error message
failed to invoke EFS utils ...
- 4,874
2
votes
0
answers
25
views
How to convert epoch to datetime in Datadog dashboard?
I have a Datadog dashboard displaying the metrics we get for our AWS Glue Zero-ETL integrations. One of those is lastSyncTimestamp, the epoch timestamp until which source has been synced to target.
I ...
- 21
-1
votes
0
answers
53
views
Spring boot reading parameters from AWS parameter store without credentials on application.yml
I'm attempting to read a parameter from aws parameter store in my spring boot application.
I succeeded on my local machine but I want to make this function without having to supply my access-key and ...
2
votes
0
answers
38
views
How should the knowledge base search operation be handled within an AWS Bedrock Agent custom orchestration Lambda?
According to the official AWS Bedrock Agent documentation, I can write a Lambda function to perform custom orchestration, also known as the ReWoo approach, in order to reduce the number of FM (...
- 21
1
vote
0
answers
42
views
Unable to pass Docker image to run to Ray Job Submit
I pulled this image - rayproject/ray:2.51.0.801bd7-extra-py310 from dockerhub with podman (ray prefers podman internally).
On AWS-EC2 then I started a head instance with - Deep Learning Base AMI with ...
- 33
1
vote
0
answers
33
views
Django Celery Beat SQS slow scheduling
Beat seems to be sending the messages into SQS very slowly, about 100/minute.
Every Sunday I have a sendout to about 16k users, and they're all booked for 6.30pm. Beat starts picking it up at the ...
- 11
1
vote
0
answers
20
views
How to troubleshoot OpenSearch upgrade eligibility failure
I'm trying to migrate an OpenSearch 2.9 cluster to OpenSearch 3.1 but the eligibility test informs the following message:
The following indices are not compatible with the target Elasticsearch ...
- 158
0
votes
0
answers
31
views
AWS S3 SCP Policy to Prevent Disabling Public Access Block but Allow Re-enabling
I need to create an AWS IAM policy that prevents users from disabling the "Block all public access" configuration on S3 buckets, but still allows them to re-enable it if it was already ...
- 1,146
0
votes
0
answers
32
views
+100
AWSSDK.S3 causes x-amz-content-sha256 error against minio
I'm using the AWSSDK.S3 for .NET (v4.0.9.1) in my app. For my tests I run minio in a docker container. I know this worked in the past but with an update to the current versions of the SDK and the ...
- 6,718
1
vote
0
answers
39
views
How to identify which IdP a user used to sign in during AWS Cognito PreAuthentication trigger?
How can I determine which Identity Provider (IdP) a user used to sign in during the AWS Cognito PreAuthentication trigger, when multiple federated IdPs (SAML/OIDC) are linked to the same user?
In the ...
- 11
0
votes
0
answers
22
views
Concurrency issue while updating micro-frontend manifest to s3
All MFEs are deployed using a single shared GitHub Actions workflow.
This workflow handles deployment for any MFE that has been updated and updates a shared manifest file that lists all deployed MFE ...
- 69
0
votes
0
answers
40
views
Batch unprocessed items are actually processed
I'm implementing a solution with high throughput, 500 - 700 items per second per instance, and I'm using DynamoDB for persistence. For every item I performat least 2 writes and one read, some like:
...
- 491
0
votes
0
answers
91
views
Sudden CPU spike on AWS RDS (PostgreSQL) after running fine for the entire last month
I encountered an issue where our AWS RDS (PostgreSQL) instance suddenly experienced a high CPU spike, even though it had been running normally for the entire previous month.
Currently, the system is ...
- 13
0
votes
0
answers
39
views
Spark Structured streaming job failed in mid of day due to delta file not found in S3 checkpoint folder
I'm using spark version 3.4.4 and running spark structured streaming application.
Data pipeline reads data from kafka continuously and do some aggregations on stream data and stores the result into ...
- 1,171
0
votes
0
answers
37
views
Setup Google Tag Gateway with AmazonLoadBalancer
I have been tasked with setting up Google Tag Gateway for the website at work, and am struggling to understand how to get this done. We use rails backend and ALB and Route53 to route requests to our ...
- 188
1
vote
0
answers
44
views
Unable to complete Amplify v6 Cognito OAuth flow in my Next.js App
I am using Amplify and Cognito's hosted UI Login page to handle my user authentication. I am able to redirect to the cognito hosted page, enter my user credentials, and redirect page to my sign in ...
- 11
1
vote
0
answers
49
views
Is there an API to check for currently supported runtimes by AWS Lambda?
I’m developing a tool to speed up the AWS Lambda creation process for my colleagues and I, as we use it extensively, and I need to periodically check for deprecated runtimes. The idea is to create a ...
- 664
-1
votes
0
answers
26
views
Expected X-Cache value when serving requests for non-existent files?
I have configured CloudFront and S3 using OAI and it generally serves files as expected.
I am trying to reduce my S3 API costs and I think about 10% of it is linked to requests for non-existent files. ...
- 4,894
0
votes
0
answers
53
views
AWS Code Deploy command not found for pm2 and pnpm
I'm deploying Node.js based server on EC2 via code deploy.
I'm using pnpm and pm2, when execution of shell script from appsepc.yml, it seems path not works. pnpm command not found or pm2 command not ...
- 355
0
votes
0
answers
53
views
AWS Bedrock Calling retrieve_and_generate from boto3 always returns "Sorry, I am unable to assist you with this request."
I am calling the retrieve_and_generate of the Bedrock API to do RAG on a set of documents. Given the following code, the generative model consistently returns:
"Sorry, I am unable to assist you ...
- 1,972
0
votes
0
answers
77
views
How to Check if a Query Touches Data Files or just Uses Manifests and Metadata in Iceberg
I created a table as follows:
CREATE TABLE IF NOT EXISTS raw_data.civ (
date timestamp,
marketplace_id int,
... some more columns
)
USING ICEBERG
PARTITIONED BY (
marketplace_id,
...
- 2,781
1
vote
0
answers
61
views
Polars LazyFrame sink_parquet + PartitionByKey slower to S3 than local disk
I'm wondering why I'm seeing such poor performance when writing a LazyFrame using PartitionByKey to S3 when compared to other methods. Here is a simple test script that writes out some random data to ...
- 266
1
vote
0
answers
32
views
NiFi ListS3 Processor SdkClientException: Name or service not known for S3 bucket
I'm setting up an Apache NiFi flow using the ListS3 processor to scan an AWS S3 bucket, but I keep getting errors, and the processor isn't listing any files from the bucket.
NiFi Processor ...
- 61
0
votes
0
answers
30
views
Inconsistent NLU for ToggleController friendlyNames
We’re building an Alexa Smart Home Skill for OmniBreeze tower fans (model DC2313R-G). Both devices are discovered with identical capabilities, including a ToggleController for oscillation using ...
- 1
0
votes
0
answers
42
views
AWS ALB - Entra OIDC - Access Token - Custom API Scopes Issue
The request flow for my setup is that first, the requests go to AWS ALB, then authentication will be done by Entra at ALB level and then if auth is successful, then the request with tokens set as ...
1
vote
0
answers
67
views
Clojure problems in resolving .local server address inside private network
First of all thanks in advance for any help, this is the first time I'm setting up a system like this, so a few points are not clear and might be written in bad form (and that's why I'm here).
...
- 31.8k
0
votes
0
answers
51
views
Pyiceberg - S3tables and boto3 session
In Pyiceberg 0.10.0, it is now possible to use a botocore session for a rest catalog, so:
import io
import os
import pandas as pd
import pyarrow as pa
from boto3 import Session
from pyiceberg....
- 485
1
vote
0
answers
55
views
Why does botocore raise ValidationException with pydantic BaseModel output constraint on variable length output types?
I am using pydantic AI and use pydantic BaseModel to set the output_type of the answer. From time to time I encounter a ValidationException. I cannot accurately pinpoint the issue creating this ...
- 117
0
votes
0
answers
42
views
Not able to send sms using AWS End User Messaging
Below are india Specific In fo , we filled in Sender Id Registration in AWS EUM. We got apporval from Kqarix Mobile, and for rest of TE we filled 00000000000000000000 as it wan bot accepting NA or &...
- 3,237
0
votes
0
answers
54
views
API Gateway not forwarding real path to Lambda
I have a Lambda (Go) behind an HTTP API Gateway.
Route /webhook is set correctly (payload format 2.0, not $default), but every request to /webhook still triggers my root / handler, im not getting the ...
0
votes
0
answers
54
views
aws ec2 ubuntu : why a command works with terminal BUT not with user_data?
i lanched an ec2 instance (ubuntu), and i have a script which looks like :
#!/bin/bash
# works fine (even directly in terminal or via user_data)
hostnamectl set-hostname ${hostname}
# works fine as ...
- 275
0
votes
0
answers
51
views
Unexpected Write Behavior when using MERGE INTO/INSERT INTO Iceberg Spark Queries
I am observing different write behaviors when executing queries on EMR Notebook (correct behavior) vs when using spark-submit to submit a spark application to EMR Cluster (incorrect behavior).
When I ...
- 2,781
1
vote
0
answers
34
views
Frequent RedisTimeoutException in .NET 9 running on Linux containers with Redis on AWS (v7.0.7)
I am frequently encountering RedisTimeoutException errors in my application. The application is built using .NET 9 and runs in Linux-based containers. Redis is hosted on AWS Elastic Cache with version ...
1
vote
0
answers
40
views
Logging username that generates GET/PUT pre-signed url for audit in s3 boto
trying to log the username that generates a presigned url for 'GET' method for auditing.
Need the info to be available server-side.
What I've tried:
Concat the username to the url '&', this wont ...
- 21
0
votes
0
answers
37
views
FastAPI: PDF upload works when called directly inside Kubernetes or locally, but results in blank files when routed through API Gateway/NGINX
I’m running a FastAPI service in Kubernetes that accepts multipart/form-data uploads and stores files in S3.
The upload endpoint works fine when I call it directly inside the cluster using the service ...
- 3
0
votes
0
answers
50
views
Linker error when importing AWS libraries in Unreal Engine
I am trying to get users to be able to log in via a Cognito account and use those credentials to make calls on a backend API. To that end I have been trying to import the Cognito libraries in an ...
0
votes
0
answers
49
views
AWS Lex + Lambda + API Gateway integration — request stuck on “Thinking…” and Lambda not triggered
I’ve built a simple web-based chatbot (called Engineer’s Toolkit) that uses Amazon Lex, AWS Lambda, and API Gateway, with a static frontend hosted on S3.
The flow is supposed to be:
Frontend (S3) → ...
0
votes
0
answers
121
views
Why am I getting "Invalid initialization vector" when calling my signed URL upload API in Node.js?
I’m working on a Node.js backend that generates a signed URL for uploading files to AWS S3 via CloudFront.
However, when I hit the signed URL endpoint from curl, I always get the following response:
{&...
0
votes
0
answers
48
views
Can't get AWS Lambda Powertools dynamic routes to work
from aws_lambda_powertools.utilities.typing import LambdaContext
from aws_lambda_powertools.event_handler import APIGatewayHttpResolver
from aws_lambda_powertools.logging import Logger
from validate ...
0
votes
0
answers
63
views
Telegram Bot sendDocument fails with Amazon S3 URL
I am using PHP with Guzzle to send a document via Telegram Bot API using sendDocument.
The file is hosted on Amazon S3 with a URL like:
https://telegramfile.xxx.amazonaws.com/file/preview/chat/...
- 1
0
votes
0
answers
19
views
awsEndpointRequestId not being consumed by headers for HTTP APIGateway
According the Customize HTTP API access logs, the logging variable $context.awsEndpointRequestId is populated from x-amz-request-id or x-amzn-requestId header.
I am directly hitting the my HTTP ...
- 469
0
votes
0
answers
52
views
Can't get a row from dynamoDB by id
I’m seeing a weird intermittent read issue with DynamoDB using the AWS SDK for JavaScript v3. For some (but not all) item IDs, the first call to GetItem returns no item, and the second call ...
- 603
0
votes
0
answers
51
views
Getting an update when trying to change Drive letters with powershell
So I'm trying to create an AMI in AWS EC2 that can be spun up by our developers. It needs to take user-data, and to do this, I had to shutdown the instance with sysprep before I could take an AMI of ...
- 435
0
votes
0
answers
21
views
AWS Amplify signInWithRedirect integration with Azure AD (Entra ID) App Registration. Can't set Application ID URI to "urn:amazon:cognito:..." anymore
To integrate my apps with Active Directory I add the following to amplify/auth/resource.ts:
externalProviders: {
saml:
{
name: 'MyApp',
...
- 441
0
votes
0
answers
45
views
Redshift - SQL Error: Internal jdbc driver error
I am trying to use for loop within the stored procedure to retrieve the employee id and process further but getting an error:
Error: SQL Error: Internal jdbc driver error
Procedure:
create or replace ...
- 7,335
0
votes
0
answers
39
views
Migrate connection settings in S3 SDK v2
I have this AWS Java V1 client configurations:
ClientConfiguration cc = new ClientConfiguration();
cc.setConnectionMaxIdleMillis(10000);
cc.setUseTcpKeepAlive(true);
cc....
- 1,252
0
votes
0
answers
36
views
AWS bedrock agent - formatting issues
I have this promp template:
# System Prompt Instructions
You are "Compliance Copilot", a structured JSON API assistant specialized in compliance analysis.
Your task: Given any user query, ...
- 5,107
Federico Mo...
scored 10
asking questions this week
Ashish Kamble
scored 15
answering questions this week
Members can contribute articles
Simply submit a proposal, get it approved, and publish it.
See how the process works
Simply submit a proposal, get it approved, and publish it.
See how the process works