https://prefect.io
Join Slack
What am I doing incorrectly? :smile:
# ask-community
j
What am I doing incorrectly? šŸ˜„
āœ… 1
n
Hey @James Phoenix when you run your flow, how do you do it?
j
I am currently testing locally just by running python3 file.py
I tried awaiting the function by putting it in asyncio.run(anaylse_youtube_video) but no luck there
n
did you call 
asyncio.run(anaylse_youtube_video)
or 
asyncio.run(anaylse_youtube_video())
?
j
Copy code
@flow
async def analyse_youtube_video():
    # 1. Download the video from YouTube and upload it to GCS
    gcs_block = GCS.load("buffalo-raw-video-files")

    # Make some fake JSON data:
    data = {"name": "John", "age": 30, "city": "New York"}
    return await gcs_block.write_path(
        "video.mp4", data)

asyncio.run(analyse_youtube_video)
This triggers:
Copy code
ValueError: a coroutine was expected, got <prefect.flows.Flow object at 0x7f952f49a3a0>
n
that makes sense, you need to give 
asyncio.run()
a coroutine like 
asyncio.run(analyse_youtube_video())
j
Cool that ran šŸ‘
How do I fix this write error?
Copy code
async def analyse_youtube_video():
    # 1. Download the video from YouTube and upload it to GCS
    gcs_block = await GCS.load("buffalo-raw-video-files")

    # Make some fake JSON data:
    data = {"name": "John", "age": 30, "city": "New York"}
    return await gcs_block.write_path(
        "video.mp4", json.dump(data))

asyncio.run(analyse_youtube_video())
I pip installed gcfs which fixed the path error, now Iā€™m running into: 
TypeError: a bytes-like object is required, not 'str'
n
GCS.write_path
accepts a string 
path
argument and a 
content
argument of type bytes it looks like you're trying to pass 
json.dump(data)
as 
content
, which won't work because 
json.dump
is for writing to files and doesn't return 
bytes
. you could replace 
json.dump(data)
with 
json.dumps(data).encode('utf-8')
j
I can confirm this works.
I do get this message though:
Copy code
UserWarning: Block document has schema checksum sha256:1daf535b60cf9304103bcb4015280a30fd6bdd9686aa0c059eb9715082b5b3ec which does not match the schema checksum for class 'GCS'.
Thanks @Nate šŸ‘
n
Don't worry about that warning! it just means that behind the scenes the implementation of the GCS block class has changed, it doesn't reflect anything you did wrong - we'll be silencing this warning soon glad I could help!
d
Iā€™m trying to do something similar, and following your example it seems to get stuck during the 
write_path
call forever with no error:
Copy code
def test_upload():
    gcs_block = GCS.load("gcs-block")
    content = "abcd".encode('utf-8')

    asyncio.run(gcs_block.write_path("test.txt", content))

test_upload()
Any ideas what could be the cause here, or even ideas on how to debug this?
Okay I have no idea what was causing it to hang, but I restarted my computer and now it works šŸ˜•
j
Change def test_upload() to 
async def test_upload():
Then await 
gcs_block.write_path("test.txt", content)
rather than using asyncio.run() inside of it
Wrap test_upload with a @task or @flow
Then call asyncio.run() outside of your flow/task
@Dominik Wagner I decided to just write some of my own Google Cloud Storage CRUD anyway instead of using a block:
gcs_crud.py
gcs_crud.py
d
Works either way - I was previously using the native google upload too, but now relying on the storage block to point to the correct bucket - figured I can still use the google functions and get the bucket with 
GCS.load(block).bucket_path
šŸ‘
šŸ™Œ 1
āœ… 1
j
šŸ‘
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by