11,268 questions
Best practices
0
votes
0
replies
24
views
Optimal way to upload large files (up to 10GB) in GCS from frontend client
My use case is simple in nature.
I have a platform where users can upload any files up to 20GB.
My current solution is:
Frontend Client asks for a presignedURL which the Backend generates
return ...
- 53
1
vote
0
answers
24
views
Is it possible to do offline upload using GCS Bucket Presigned URL resumable vs write
I have a Frontend Client which lets users uploads any number of files of any size (think up to 100 GB File).
Currently I am using GCS buckets presigned URL to upload the file. My current ...
- 53
0
votes
0
answers
26
views
Firebase CLI fails with "An unexpected error has occurred" only when deploying storage
I'm trying to deploy Cloud Storage CORS settings using the Firebase CLI, but it consistently fails with a generic "Error: An unexpected error has occurred."
Project ID:
oa-maintenance-v2
...
- 1
0
votes
0
answers
35
views
Compute Engine image creation fails with “Required ‘read’ permission for storage object of a bucket” from Java SDK, works via REST API and Console
I am trying to create a virtual image from a tar file that is present in the storage bucket
When tried from my Java SDK code which is my actual requirement
I get this
Required ‘read’ permission for ‘${...
2
votes
1
answer
74
views
How to save Google Cloud GET Object API mediaLink response to local storage?
I wrote this code that uses Google's Cloud API to get an object from my bucket and download it. It works perfectly when I had my bucket set to public (allUsers added to Principal w/ all the required ...
- 31
0
votes
1
answer
69
views
distcp creating file in GCP bucket instead of file inside directory
Context:
using distcp, I am trying to copy HDFS directory including files to GCP bucket.
I am using
hadoop distcp -Dhadoop.security.credential.provider.path=jceks://$JCEKS_FILE hdfs://nameservice1/...
- 49
1
vote
2
answers
89
views
ModuleNotFoundError: No module named 'google' in WSL
I was trying to use google cloud storage in a python virtual environment. I tried installing google-cloud-storage and whenever I run the code I always get the error ModuleNotFoundError: No module ...
- 52
0
votes
1
answer
82
views
GCSToGCSOperator is moving folder along with files when using option move_object as True
I have a requirement to move files from a source folder to destination folder in different GCS buckets. I am using GCSToGCSOperator with following config:
source_bucket: "source_bucket"
...
- 1,936
1
vote
0
answers
180
views
Running huggingface models offline using huggingface snapshot download causes errors
I have been trying to run some models from huggingface locally.
The script is being hosted on google cloud run.
Since running the instance multiple times triggers rate limiting, I have downloaded the ...
- 13
0
votes
0
answers
42
views
Firebase storage image urls keep returning access denied response
I have a small app where I am using firebase functions to upload an image into firebase storage.
Once done, I store this image url against an object in firebase db and then reuse this image in the app ...
- 1
1
vote
0
answers
95
views
How can I authenticate and upload files to Google Cloud Storage in Ballerina without manually generating access tokens?
I'm trying to upload files to a Google Cloud Storage bucket from a Ballerina application.
Right now, the only way I’ve found to authenticate is by manually generating an access token using a service-...
0
votes
1
answer
66
views
How to prevent Django from generating migrations when using dynamic GoogleCloudStorage in a FileField?
On my Django Project I have a model that a property is used to store videos in a specific and single Google Cloud Storage bucket, using FileField.
The model is defined like this:
from storages....
- 851
0
votes
1
answer
73
views
How to limit size of upload using Google Cloud Signed URL
I'm using Google Cloud Storage Signed URLs with the NodeJS SDK to allow my client app to directly upload to Cloud Storage.
I want to limit the size of the file the client is allowed to upload.
How do ...
- 1,467
0
votes
1
answer
75
views
Cannot read properties of undefined (reading 'cloudStorageURI') Generation 2 Google Cloud Storage Function
I am migrating a generation 1 node.js google cloud functions to generation 2 cloud run.
It uses a onDocumentCreated function to create and save a file to cloud storage built from a collection of data ...
- 135
1
vote
1
answer
93
views
how to unit test flask method using google cloud storage library
Im trying to create unit tests for this method but im getting the error saying credentials not found
from io import BytesIO
from google.auth.exceptions import DefaultCredentialsError
from google....
