Tyler Wanner
03/13/2020, 5:10 PMTyler Wanner
03/13/2020, 5:32 PMTyler Wanner
03/13/2020, 5:33 PMJoe Schmid
03/13/2020, 7:34 PMdocker run hello-world
succeeds. (There is special "docker-in-docker" config we use with Screwdriver and this works well for many of our other projects.) I can also build Docker storage successfully on my laptop from inside the same image that our CI/CD system is using. I'm sure it's some simple config that I'm missing -- anybody run into this?
15:25:50 Traceback (most recent call last):
15:25:50 File "/usr/local/lib/python3.7/runpy.py", line 193, in _run_module_as_main
15:25:50 "__main__", mod_spec)
15:25:50 File "/usr/local/lib/python3.7/runpy.py", line 85, in _run_code
15:25:50 exec(code, run_globals)
15:25:50 File "/usr/local/lib/python3.7/site-packages/srm_datasci-0.1.0-py3.7.egg/srm/datasci/flows/util/test_flow.py", line 35, in <module>
15:25:50 flow.register("Utilities")
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1412, in register
15:25:50 no_url=no_url,
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/client/client.py", line 616, in register
15:25:50 serialized_flow = flow.serialize(build=build) # type: Any
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/core/flow.py", line 1221, in serialize
15:25:50 storage = self.storage.build() # type: Optional[Storage]
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 263, in build
15:25:50 self._build_image(push=push)
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 293, in _build_image
15:25:50 self.pull_image()
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 483, in pull_image
15:25:50 client = self._get_client()
15:25:50 File "/usr/local/lib/python3.7/site-packages/prefect/environments/storage/docker.py", line 471, in _get_client
15:25:50 return docker.APIClient(base_url=self.base_url, version="auto")
15:25:50 File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 190, in __init__
15:25:50 self._version = self._retrieve_server_version()
15:25:50 File "/usr/local/lib/python3.7/site-packages/docker/api/client.py", line 215, in _retrieve_server_version
15:25:50 'Error while fetching server API version: {0}'.format(e)
15:25:50 docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
itay livni
03/14/2020, 4:00 AMBob Colner
03/15/2020, 11:06 PMbardovv
03/16/2020, 7:10 AMbardovv
03/16/2020, 8:02 AMbardovv
03/16/2020, 8:03 AMJohn Ramirez
03/16/2020, 2:15 PMNathan Molby
03/16/2020, 2:39 PMJohn Ramirez
03/16/2020, 7:42 PMScott Zelenka
03/17/2020, 8:05 PMprefect.utilities.exceptions.ClientError: Flows with required parameters can not be scheduled automatically.
I understand what the error message is saying, but in the parameter_defaults
for the InvervalClock
of the Flow, I have the required Parameters defined. It's easy to remove the required flag for these Parameters, but for code readability I'd like to keep them in there.
Why is it considered an error, when all of the IntervalClock
defined in the Schedule
of the Flow have a value for the required Parameters defined?Tyler Wanner
03/17/2020, 8:50 PMScott Zelenka
03/17/2020, 9:17 PMMarch 17th 2020 at 5:13:44pm | prefect.CloudTaskRunner
INFO
Task 'sfdc_download_data': Starting task run...March 17th 2020 at 5:13:44pm | prefect.CloudTaskRunner
ERROR
Unexpected error: KeyError('text')
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/prefect/engine/runner.py", line 48, in inner
new_state = method(self, state, *args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/prefect/engine/cloud/task_runner.py", line 225, in check_task_is_cached
for key, res in (candidate_state.cached_inputs or {}).items()
File "/usr/local/lib/python3.7/site-packages/prefect/engine/cloud/task_runner.py", line 225, in <dictcomp>
for key, res in (candidate_state.cached_inputs or {}).items()
KeyError: 'text'
Zachary Hughes
03/17/2020, 9:37 PMbardovv
03/18/2020, 6:50 PMfrom prefect import Flow, task
@task
def add_ten(x):
return x + 10
with Flow('simple map') as flow:
mapped_result = add_ten.map([1, 2, 3])
I am using this code expected output is 11,12,13 in mapped_result how do I print the result?bardovv
03/18/2020, 6:58 PMLuke Orland
03/18/2020, 7:57 PMMarch 17th 2020 at 2:32:51am | agent
ERROR
An error occurred (ThrottlingException) when calling the RunTask operation (reached max retries: 4): Rate exceeded.
bardovv
03/19/2020, 8:47 AMemre
03/19/2020, 10:02 AMShellTask
, in which a git clone
command doesnāt return some of the stdout lines (despite using return_all=True
). The command is as follows: git clone https://{some_url}.git dbt_project -v -b master --depth 1
.
Running it from bash logs the following to stdout:
Cloning into 'dbt_project'...
POST git-upload-pack (156 bytes)
POST git-upload-pack (165 bytes)
remote: Counting objects: 40, done.
remote: Compressing objects: 100% (34/34), done.
remote: Total 40 (delta 2), reused 0 (delta 0)
Unpacking objects: 100% (40/40), done.
While running the same command through ShellTask
, and iterating over the resulting list of stdout lines logs the following:
[2020-03-19 09:57:50,792] INFO - prefect.Task: print_task | Cloning into 'dbt_project'...
[2020-03-19 09:57:50,792] INFO - prefect.Task: print_task | POST git-upload-pack (168 bytes)
[2020-03-19 09:57:50,793] INFO - prefect.Task: print_task | POST git-upload-pack (177 bytes)
By any means, I am not well versed on bash, and this may very well be an issue about how git clone
logs to stdout. Wondering if anybody else encountered a similar issue, and if they managed to come up with a fix?Eric
03/19/2020, 12:43 PMBrett Naul
03/19/2020, 4:37 PMMatt Juszczak
03/19/2020, 10:39 PMArsenii
03/20/2020, 2:55 AMbardovv
03/20/2020, 12:58 PMAdam Roderick
03/20/2020, 3:18 PMBraun Reyes
03/20/2020, 3:40 PMJeremiah
Daniel Manson
03/20/2020, 7:37 PM