Questions tagged [amazon-s3]

Amazon S3 (simple storage service) is an online object storage service from Amazon Web Services.

Filter by
Sorted by
Tagged with
0
votes
0answers
15 views

(boto3) How to programmatically give s3 bucket permission to invoke lambda?

What I am doing Currently I am trying to dynamically create an event on a bucket to call a lambda function, whenever a trigger like s3:ObjectCreated:* or s3:ObjectRemoved:* occurs. My current code ...
0
votes
1answer
11 views

Invalid according to Policy: Extra input fields

I have the following code: var RNS3 = exports.RNS3 = function () { function RNS3() { _classCallCheck(this, RNS3); } _createClass(RNS3, null, [{ key: 'put', value: function put(file,...
0
votes
0answers
16 views

How to use select_object_content via rusoto / rust?

The following code tries to select some data from a file stored on S3: let client = S3Client::new(Region::default()); let source = ... object providing bucket and key ...; let r = ...
0
votes
0answers
8 views

Example for s3 ListAllMyBuckets for Python WITHOUT SDK [on hold]

Example of code en Python WITHOUT SDK for Python I am looking for a code example like here https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html for s3: ...
0
votes
0answers
8 views

How to redirect different conditions with the same subdirectory?

I'm having problem in s3 redirect rules to redirect different conditions using the same subdirectory. It looks like just one condition is being triggered. <RoutingRules> <RoutingRule> ...
0
votes
0answers
17 views

get HTTP/1.1 403 Forbidden when trying to access private content stored in S3(static web hosting) using cloudfront and privateKey

I am receiving following response from AWS CloudFront from Java application where I am trying to access a private content (web page) using signed cookies. HTTP/1.1 403 Forbidden [Content-Type: ...
1
vote
2answers
19 views

AWS S3 Access Denied when open objetc url in browser

I am new to aws.. I uploaded a image in the bucket if i try to open the object url in browser im getting the following error. below are the bucket policies i have configured my bucket with { "...
0
votes
0answers
10 views

Track S3 File upload progress from GraphQL Mutation

Im currently trying to upload a file with help of the documentation (Android). For that I am using the complex object way, so I have a dynamoDB entry and the S3 File upload at once. So I am not using ...
0
votes
3answers
33 views

Delete Folders, Subfolders and All Files from a S3 bucket older than X days

I have a S3 bucket with the below architecture - Bucket |__2019-08-23/ | |__SubFolder1 | |__Files | |__2019-08-22/ |__SubFolder2 I want to delete all folders, subfolders and files within ...
0
votes
0answers
5 views

dask export dataframe to remote storage (S3)

When using dask_df.to_csv('s3://mybucket/mycsv.csv') I get an error that I should install s3fs I did install it in the workers (with client.run()) and still got the error. So I installed s3fs ...
0
votes
1answer
29 views

Uploading image to presigned url using nodejs

I am trying to upload a image to url using xmlhttprequest module in nodejs.For that I need to append fields to form data. const formData = new FormData(); //op.data->presignedData ...
-3
votes
0answers
28 views

How do I read/write multiple objects concurrently from an S3 bucket using Java? [on hold]

I need to be able to read a large number (10K - 100K+) of small (2-3KB) objects from a bucket prefix, process them and then write the response back to another prefix in the bucket before they move ...
0
votes
1answer
19 views

How to get S3 bucket's full URL for firewall?

I am using azcopy (Azure Copy) to copy data from an S3 bucket thru a proxy. We cannot allow *.amazonaws.com or any wildcard. The security team requires a rule for each bucket to prevent exfiltration ...
0
votes
0answers
9 views

Keep query strings with AWS Amplify Hosting - React

I currently have a react app using react-router-dom that works locally but not when deployed using AWS Amplify. The problem is that when I route to a URL and use query strings, the browser gets a ...
2
votes
0answers
20 views

How do I specify a FIPS endpoint url in a boto3 S3 client configuration?

I am currently attempting to configure my boto3 client to connect using a custom endpoint URL for FIPS. https://aws.amazon.com/compliance/fips/ I have the following code that works with boto3 for EC2....
0
votes
1answer
20 views

PHP S3 SDK upload object with versioning turned off

As I'm crusing through the Amazon S3 SDK for PHP, after some fun with $this->s3Instance->putObject() I now will want to delete these objects. Yet when I run $this->s3Instance->...
0
votes
2answers
26 views

Download millions of records from s3 bucket based on modified date

I am trying to download millions of records from s3 bucket to NAS. Because there is not particular pattern for filenames, I can rely solely on modified date to execute multiple CLI's in parallel for ...
0
votes
0answers
13 views

Export DynamoDB data to S3 in JSON format with scheduler

I need to export complete Dynamo Db tables(which has millions of records) to S3 in Json format(these json files are later used for Athena). It must also include a cron job which is needed to run ...
1
vote
1answer
19 views

Do I need the code deploy step in aws code pipeline for a static s3 website

I created a repo in Code Commit for a static s3 website Then I created a CodePipeline and configured the code build part. There I set the Build Spec file with the some basic commands: build and then ...
0
votes
0answers
13 views

Problem using localstack to invoke lambda from s3

I'm using localstack and trying to get s3 invocations of my lambda working. I can invoke the lambda from the cli successfully, and s3 seems to be trying, but nothing happens. I created my lambda ...
0
votes
0answers
12 views

com.ibm.cloud.objectstorage.services.s3.model.AmazonS3Exception:Invalid Grant

I am trying to copy objects from bucket in ibm object storage from source client to target client. but while granting grant of source client owner to target client Acl iam getting this error. Also is ...
0
votes
1answer
22 views

ConnectionRefusedError: [Errno 61] Connection refused & Could not connect to the endpoint URL: http://127.0.0.1:3001

I am trying to run integration tests using boto3 and aws. It seems the tests are unable to connect to the localhost endpoint so the tests are failing. I'm quite sure I've configured my aws environment ...
-2
votes
1answer
34 views

NodeJS Lambda function: how do I parse and detect a key inside an event

I am using Lambda-local to test my event, which will simulate a JSON file being sent to my S3 bucket and ultimately that i need to save. Here is my event.json: { "Records":[ { "s3"...
0
votes
0answers
30 views

signed URL to post a file to S3 [duplicate]

I'm completely stumped on creating a static web page to post a file to S3. I've used the sample code I found from this page (Link: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example....
0
votes
0answers
7 views

Displaying a PDF from AWS S3 in WebView causes download instead of displaying inline

When I try to view a PDF from an S3 bucket in a WebView, it downloads even though the Content-Type is set to application/pdf and the Content-Disposition to inline (tried without it as well) in the AWS ...
0
votes
1answer
32 views

How to use Rscript with readr to get data from aws s3

I have some R code with readr package that works well on a local computer - I use list.files to find files with a specific extension and then use readr to operate on those files found. My question: ...
0
votes
1answer
36 views

Getting undefined after parsing the data in node js

I am trying to upload an image to S3 from a AWS lambda function. For this I have followed this tutorial - link. I have created an post API using API Gateway and written nodejs code in lambda to post ...
-2
votes
0answers
22 views

How to read the content of a file without downloading it from an AWS S3 bucket [on hold]

I'd like to read the content of a text file located in an S3 bucket without downloading it with a java method I found the solution: public class S3Sandbox { public static void main(String[] ...
0
votes
2answers
24 views

Create a folder in AWS S3 bucket with current date

I want to create a directory with current date as the directory name and move files created on my local file system into this new directory. The goal is to transfer files created everyday in local ...
0
votes
1answer
19 views

ArgumentError: missing required option :key

I am trying to get an object from an S3 bucket via: s3 = Aws::S3::Resource.new( region: ENV['AWS_REGION'], access_key_id: ENV['AWS_ACCESS_KEY_ID'], secret_access_key: ENV['...
0
votes
0answers
17 views

How to live stream from android device camera to store video chunks on Amazon S3 bucket?

I'm working on a project for an Android solution around video recording and analysis. The idea is that since the video would be huge in size, so I have to design a solution wherein say every 30 ...
0
votes
3answers
40 views

Delete all keys in a bucket of s3 which are three days old using python

How do I delete all the keys in an S3 bucket which are three days old using python? S3 bucket contents are like: mybucket001/backup/1566394660_21_08_2019_backup mybucket001/backup/...
1
vote
1answer
15 views

Reading multiple CSV files object at once from AWS S3 using aws.s3 package

I need to read multiple csv files from AWS S3 bucket with aws.s3 package in R and finally combine those files in single dataframe for further analysis. Let' say I have several files in my S3 bucket ...
0
votes
0answers
23 views

'ValueError: Required parameter not set' Django app following AWS configuration and Heroku deployment

I have just deployed my Django app to Heroku, having configured the app to store media files in an AWS S3 bucket. Everything was working correctly in the development server, but when I try to access ...
-1
votes
1answer
25 views

Not able to create logs in S3 bucket using a Lambda function

From my Lambda Python code, I'm trying to create a log to my S3 bucket (s3://my_bucket/logs/) and throwing an error. It is working fine when I execute outside & generating logs. Error is below: *...
0
votes
1answer
25 views

Using PDFKit to store a PDF in S3 on the fly

I'm trying to create a pdf of some images (in datauri format) is nodejs and having the pdf stored in my S3. The return of the function is expected to provide the s3 URL of the file. I'm using parse-...
0
votes
0answers
11 views

Deleting a s3 directory from terminal is not getting reflected in metadata

I have a spark program and its output directory is in s3. When I delete the previous output directory from terminal using aws s3 rm --recursive for next execution of spark program, an error is ...
1
vote
2answers
38 views

How to remove a file with weird name on AWS S3?

My colleague accidentally created a file named U00d89db4dc52b87f205fb151f6225b39&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&#xf;&...
0
votes
0answers
27 views

How do I remove GPS data from image/video in Javascript or Node.js?

In my application users can upload images and videos that are publicly available - profile pictures, blog posts, etc. Since they are publicly available to other users, then that means the metadata ...
0
votes
0answers
14 views

How do I use a systemd service using aws ec2 to run a minecraft server?

So i've been trying to start a server on aws ec2 using the following script: #!/usr/bin/env bash sudo yum -y update sudo yum -y install java sudo aws s3 sync s3://minecraftdirectory/minecraft/ /...
0
votes
1answer
23 views

aws s3 ls - Bash script issue

I am executing a bash script that includes the following lines source scl_source enable rh-python36 AWS_ACCESS_KEY_ID=$accessKey AWS_SECRET_ACCESS_KEY=$secretKey aws s3 ls When the second line of ...
-1
votes
1answer
25 views

Getting error “The AWS Access Key Id you provided does not exist in our records” when trying to sync

I am trying to download all the available files from my s3 bucket to my local machine. I have installed AWS cli. and then I have used aws configure to setup access key and secret key too. I am facing ...
0
votes
0answers
33 views

Upload a CSV directly to AWS S3 without saving to disk?

I scraped a website, created a PySpark DataFrame and now I would like to upload the DataFrame in a CSV format to my AWS S3 bucket without saving to disk. I use PyCharm IDE (where I set my Access ...
0
votes
1answer
22 views

How would I display an image from s3 on a webpage in my flask app without saving it?

I have an ec2 instance running a flask website, and I want to get a certain image from s3 only if a certain user is logged into the site and display it on the homepage. The image can be downloaded ...
0
votes
1answer
18 views

How to use aws s3 sync from the CLI for more than 1000 objects

I'm attempting syncing the contents of an S3 bucket (actually digital ocean space) to my local hard drive with aws-cli s3 sync or aws-cli s3 cp --recursive I've tried using both the aws-cli sync and ...
0
votes
2answers
23 views

Incorrect S3 bucket policy is detected for bucket at Cloud Trail

Receveing this error Incorrect S3 bucket policy is detected for bucket: (Service: AWSCloudTrail; Status Code: 400; Error Code: InsufficientS3BucketPolicyException; Request ID: ebaf35b8-a38e-4357-...
0
votes
1answer
25 views

uploading file to specific folder in S3 bucket using boto3

My code is working. The only issue I'm facing is that I cannot specify the folder within the S3 bucket that I would like to place my file in. Here is what I have: with open("/hadoop/prodtest/tips/ut/...
0
votes
1answer
16 views

How to download an asset (zip file) stored on Amazon S3 without having direct access to the bucket

I'm trying to download a zip file stored in an Amazon S3 bucket. I've found the location of the file in the site's javascript but if I just stick that in the address bar i get 'access denied'. The ...
0
votes
0answers
17 views

Lambda to parse and validate JSON files and copy them onto a S3 bucket's subfolder: how to make it generic?

I have some very concrete Lambda nodejs code that does the following: Validate JSON files against a provided schema Parse the JSON files to identify a specific key Copy validated files holding the ...
1
vote
1answer
32 views

EntityTooSmall when uploading to S3

I'm trying to upload a file to my S3 bucket using multipart upload. I'm following the exact same code as found here. The problem is, whenever it reaches the last part of my file, I get a ...