https://prefect.io
Join Slack
Hey gang. Anyone seen this error before? I cannot seem to find any documentation or anything related...
# ask-community
a
Hey gang. Anyone seen this error before? I cannot seem to find any documentation or anything related to it. 
OSError: source code not available
Its for a simple hello world flow. * The flow is being run from a local jupyter notebook * The notebook connects to a prefect server running in AWS EC2 * The dask cluster is running inside AWS ECS * The dask scheduler is running on a seperate EC2 instance
n
hey @Ansaf! can you show the example? also if you would, what's 
prefect version
for you?
a
Hey Nate, just a moment
Copy code
@task
def hello_task():
    return "Hello, World!"

@flow(task_runner=DaskTaskRunner(address="<tcp://xx.xx.xx.xx:8786>"))
def hello_flow():
    result = hello_task.submit().result()
    print(result)
The dask scheduler and the prefect server are running on the same (persistent) AWS ec2 instance. Whereas the workers are spun up as AWS ECS tasks. I can see on the dask dashboard that is exposed by the dask scheduler that the worker is indeed picking up the task.
And I am running the above flow from my local jupyter notebook, the PREFECT_API_URL is set to the prefect server running in the cloud and I can see that the task is picked up by visiting the prefect servers dashboard
Some more info: I just ran the code without prefect and that worked.
Copy code
from dask.distributed import Client
import time

client = Client("<tcp://xx.xx.xx.xx:8786>") 
def test_func():
    for x in range(4):
        print("Hello, World!")
        time.sleep(1)
    return "Done"

# Submit the function to Dask workers
future = client.submit(test_func)
result = future.result()

print(result)
@Nate bump 😢
2 Views
Open in Slack
PreviousNext
Powered by

Prefect Community

Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.

Powered by