Fixed
Status Update
Comments
em...@google.com <em...@google.com>
em...@google.com <em...@google.com>
jo...@google.com <jo...@google.com> #2
Perhaps one way to solve this would be to allow all of the settings you can set with com.google.appengine.api.files.GSFileOptions.GSFileOptionsBuilder on the com.google.appengine.api.blobstore.UploadOptions class. So you could specify up front what the filename should be or any metadata you need.
jo...@google.com <jo...@google.com> #3
If you retrieve the BlobInfo for that blob key you should be able to get the filename.
jo...@google.com <jo...@google.com> #4
I can get the BlobInfo for the key, but it just gives me the user's upload filename which is not what I need. I ran a test in prod and it shows this:
BlobInfo info = new BlobInfoFactory().loadBlobInfo(blobKey);
// This just gives the user's filename like "cute-kitten.jpg":
log.info (info.getFilename());
However, I'm looking for the autogenerated filename that is created in google storage. I refreshed the cloud storage explorer and found the GS filename generated for this test was "L2FwcHMtdXBsb2FkL2Jsb2JzL2UzNUFBVE0tMEIxM1dwREFTc0t3RW9LOFE" and I can't find a way to get this.
BlobInfo info = new BlobInfoFactory().loadBlobInfo(blobKey);
// This just gives the user's filename like "cute-kitten.jpg":
However, I'm looking for the autogenerated filename that is created in google storage. I refreshed the cloud storage explorer and found the GS filename generated for this test was "L2FwcHMtdXBsb2FkL2Jsb2JzL2UzNUFBVE0tMEIxM1dwREFTc0t3RW9LOFE" and I can't find a way to get this.
jo...@google.com <jo...@google.com> #5
This issue is not specific to Java. I'm using Python and cannot retrieve the Google storage filename after a user upload. The BlobInfo key is unrelated as far as I know (Strangely it starts with "?ISO-8859-1...").
jo...@google.com <jo...@google.com> #6
Facing the same issue. Without knowing GS filename this entire API is useless for GAE app. Its possible to upload data to GS but impossible to read/delete them. Is there any workaround for this issue?
jo...@google.com <jo...@google.com> #7
at...@gmail.com <at...@gmail.com> #8
I'm not sure the person posting the last comment actually read the issue. The problem is that we *only* have the blobkey. That's not enough for some use cases using GCS.
For example, it is impossible to add metadata headers to the uploaded files that are stored in GCS.
For example, it is impossible to add metadata headers to the uploaded files that are stored in GCS.
em...@google.com <em...@google.com> #9
I understand & acknowldge "it is impossible to add metadata headers to the uploaded files that are stored in GCS."
I was responding to comment #5 which was "Its possible to upload data to GS but impossible to read/delete them."
I was responding to
em...@google.com <em...@google.com> #10
Ok, that makes sense and I see where you are coming from.
A few clarifications... Just trying to help out the next guy that hits this, since I've since moved from GAE to AWS...
1. The documentation is unclear if you call delete on a blobkey if it deletes the key/pointer or the actual file in GCS. For some reason I assumed that the delete removed the blobkey, but not the actual file.
2. When our application was written to work with GCS, we had a custom uploader that just put files in GCS based on a naming convention into the filestore. So to read the file we didn't have to know a "key" because it was just a user's ID and some other derived info.
When we switched to the official uploader that always returned a blobkey, we couldn't do that anymore. Without changing our code to store the blobkey we would have no way to read it. Perhaps the poster from comment #5 means something similar? Just a guess...
A few clarifications... Just trying to help out the next guy that hits this, since I've since moved from GAE to AWS...
1. The documentation is unclear if you call delete on a blobkey if it deletes the key/pointer or the actual file in GCS. For some reason I assumed that the delete removed the blobkey, but not the actual file.
2. When our application was written to work with GCS, we had a custom uploader that just put files in GCS based on a naming convention into the filestore. So to read the file we didn't have to know a "key" because it was just a user's ID and some other derived info.
When we switched to the official uploader that always returned a blobkey, we couldn't do that anymore. Without changing our code to store the blobkey we would have no way to read it. Perhaps the poster from
em...@google.com <em...@google.com> #11
I meant that its impossible to access data that resides in GS (are they?) via GS REST API, since we don't know data url. I.e. if I upload data via GAE API I forced to use only this API (e.g. blobstoreService.serve() to give blob data to the user etc.).
However, this issue also adds a lot of confusion. Files are being uploaded to the GS (I can see them in GS Manager) but also appear in GAE app blobstore viewer. Some question arise here:
- will I be charged twice, for GAE app storage and GS storage space
- does it differ from the case when I put data to GS via curl -X PUT http://.../bucker/file in any way, i.e. is GS differs in any way by implementation from Blobstore, or its just a rebrended storage with new REST API to access it and different data namespace?
However, this issue also adds a lot of confusion. Files are being uploaded to the GS (I can see them in GS Manager) but also appear in GAE app blobstore viewer. Some question arise here:
- will I be charged twice, for GAE app storage and GS storage space
- does it differ from the case when I put data to GS via curl -X PUT http://.../bucker/file in any way, i.e. is GS differs in any way by implementation from Blobstore, or its just a rebrended storage with new REST API to access it and different data namespace?
em...@google.com <em...@google.com> #12
with 1.7.5 release is there any way to set the filename when calling createUploadUrl(java.lang.String, com.google.appengine.api.blobstore.UploadOptions)
Is it correct then, that to do named filed upload I have to use GCS rest api?
Is it correct then, that to do named filed upload I have to use GCS rest api?
em...@google.com <em...@google.com> #14
[Comment deleted]
jh...@themeetgroup.com <jh...@themeetgroup.com> #15
sm...@google.com <sm...@google.com> #16
I'm having the same issue with python Blobstore API and GCS. What I found is that, when using blobstore.creat_upload_url('/foo/bar/', gs_bucket_name='foobucket') to generate a upload url just like using blobstore only. It works, the data will be uploaded to GCS, but also leave an instance that you can see in the BlobViewer from application dashboard. So the first question is, does Google charge for both object (blob and gcs)?
Second, when using blobstore.create_gs_key('/gs/bucket/filename') returns a string, I guess it's a key associated with the file exsiting on GCS but it's not documented how to use it. First, I thought it is used to retrieve/delete object in GCS, but it's not! I tried blobstore.delete('key') to delete object in GCS it doesn't do anything. Meanwhile, when using get_uploads('fieldname')[0] to retrieve a Blobinfo from form, you can get BlobKey from this, i.g.:
uploaded_file = self.get_uploads('fieldname')[0]
file_blobkey = uploaded_file.key()
Then I can use this blobkey to delete object in BlobViewer and GCS, then the second question is what does create_gs_key() do? How we gonna use it?
Third question is, how do I serve files from GCS through AppEngine without using AppEngine's bandwidth? I know ImageService can serve image, does it work for pdf/csv/word etc. as well?
Need help here!
Thanks
Second, when using blobstore.create_gs_key('/gs/bucket/filename') returns a string, I guess it's a key associated with the file exsiting on GCS but it's not documented how to use it. First, I thought it is used to retrieve/delete object in GCS, but it's not! I tried blobstore.delete('key') to delete object in GCS it doesn't do anything. Meanwhile, when using get_uploads('fieldname')[0] to retrieve a Blobinfo from form, you can get BlobKey from this, i.g.:
uploaded_file = self.get_uploads('fieldname')[0]
file_blobkey = uploaded_file.key()
Then I can use this blobkey to delete object in BlobViewer and GCS, then the second question is what does create_gs_key() do? How we gonna use it?
Third question is, how do I serve files from GCS through AppEngine without using AppEngine's bandwidth? I know ImageService can serve image, does it work for pdf/csv/word etc. as well?
Need help here!
Thanks
em...@google.com <em...@google.com> #17
Android Studio Giraffe Canary 6 is available to download today:
Can you try that version and let us know if the infinite loader issue is fixed, or still occurs?
jh...@themeetgroup.com <jh...@themeetgroup.com> #18
Giraffe Canary 6 works for me on a quick test. Files open, properly linking between java and native sources.
jo...@google.com <jo...@google.com> #19
I'm resolving this bug and spinning of comment#14 into its own bug b/271119886 .
Description
Build #AI-223.7571.182.2231.9506613, built on January 19, 2023
Runtime version: 17.0.5+0-b2043.56-9484017 amd64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
Windows 10 10.0
GC: G1 Young Generation, G1 Old Generation
Memory: 4096M
Cores: 8
Registry:
external.system.auto.import.disabled=true
ide.text.editor.with.preview.show.floating.toolbar=false
ide.instant.shutdown=false
Regression: Yes (Electric Eel | 2022.1.1)
Repro: 3/3
Found in: SANITY
Platform: All
For C++ Projects:
NDK Version: 25.1.8937393
Cmake Version: 3.22.1
Steps to Reproduce:
1. Create a Native C++ project.
2. On successful build, open CPP folder -> any .cpp file in the editor.
Expected Results:
The file should the content in the editor
Observed Results:
Opened CPP file is Showing infinite loader, failing not loading the content in the editor.
In the Studio at bottom, showing "Context: Indexing" for long time.