Hi all, I wrote a small context manager, to scale out/in some EC2 containers with GPUs, so that they are only running, when an ECSTask is being executed, but otherwise are stopped to save costs. However, the
__exit__
method of the context manager is called shortly after the ECSTask starts running - so the task does not actually run. However,
__enter__
works fine, and the ECSTask execution only starts when the EC2 is up. Could anyone explain me why this is happening and how I could fix it?
Copy code
with ECSScaler(print_func=<http://logger.info|logger.info>) as scaler:
ECSTask(...).run()
z
Zanie
06/09/2023, 6:05 PM
ECSTask().run()
is an async method
Zanie
06/09/2023, 6:05 PM
Actually I guess it’s probably sync compatible and should block there. Hm.
Zanie
06/09/2023, 6:05 PM
What’s the return value of
.run()
?
p
Peter Nagy
06/12/2023, 7:46 AM
It is a
prefect_aws.ecs.ECSTaskResult
object with
status_code=0
Peter Nagy
06/12/2023, 10:05 AM
I have been testing a bit further today, and now it seems to work. My problem was (I think), passing the SequentialRunner to the Flow decorator with the wrong parameter (should be
task_runner
), which did not throw any errors, but also did not work🤦 Thanks for the quick reply though, I appreciate it! 🙂
Bring your towel and join one of the fastest growing data communities. Welcome to our second-generation open source orchestration platform, a completely rethought approach to dataflow automation.