Atsushi Saito
04/07/2022, 5:50 PMKevin Kho
04/07/2022, 5:52 PMflow.run()
. flow.run()
is for local testing and does not appear on Cloud/Server. You need to register it and run it with an agent. Check this tutorial onwardsAtsushi Saito
04/07/2022, 6:02 PM> python pref.py
[2022-04-08 02:20:16+0900] INFO - prefect.FlowRunner | Beginning Flow run for 'dvc-exam'
[2022-04-08 02:20:16+0900] INFO - prefect.TaskRunner | Task 'raw_data_path': Starting task run...
[2022-04-08 02:20:16+0900] INFO - prefect.TaskRunner | Task 'raw_data_path': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:16+0900] INFO - prefect.TaskRunner | Task 's3_url': Starting task run...
[2022-04-08 02:20:16+0900] INFO - prefect.TaskRunner | Task 's3_url': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:16+0900] INFO - prefect.TaskRunner | Task 'prepare': Starting task run...
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'prepare': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'aws_profile': Starting task run...
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'aws_profile': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'format_command': Starting task run...
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'format_command': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:18+0900] INFO - prefect.TaskRunner | Task 'ShellTask': Starting task run...
[2022-04-08 02:20:20+0900] INFO - prefect.TaskRunner | Task 'ShellTask': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:20+0900] INFO - prefect.TaskRunner | Task 'featurize': Starting task run...
The input data frame ./data/prepared/train.tsv size is (20017, 3)
The output matrix data/features/train.pkl size is (20017, 3002) and data type is float64
The input data frame ./data/prepared/test.tsv size is (4983, 3)
The output matrix data/features/test.pkl size is (4983, 3002) and data type is float64
[2022-04-08 02:20:36+0900] INFO - prefect.TaskRunner | Task 'featurize': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:36+0900] INFO - prefect.TaskRunner | Task 'train': Starting task run...
Input matrix size (20017, 3002)
X matrix size (20017, 3000)
Y matrix size (20017,)
[2022-04-08 02:20:40+0900] INFO - prefect.TaskRunner | Task 'train': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:40+0900] INFO - prefect.TaskRunner | Task 'evaluate': Starting task run...
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[1]': Starting task run...
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[1]': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[2]': Starting task run...
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[2]': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[0]': Starting task run...
[2022-04-08 02:20:41+0900] INFO - prefect.TaskRunner | Task 'evaluate[0]': Finished task run for task with final state: 'Success'
[2022-04-08 02:20:41+0900] INFO - prefect.FlowRunner | Flow run SUCCESS: all reference tasks succeeded
Kevin Kho
04/07/2022, 6:17 PMAtsushi Saito
04/08/2022, 5:46 AMtutorial
,
Terminal 1:
prefect server start --ui-port 2222
Termnal 2:
prefect agent local start
Terminal 3:
python flow.py
for the following source.
import prefect
from prefect import task, Flow
@task
def say_hello():
logger = prefect.context.get("logger")
<http://logger.info|logger.info>("Hello, Cloud!")
with Flow("hello-flow") as flow:
say_hello()
# Register the flow under the "tutorial" project
flow.register(project_name="tutorial")
terminal 3 shows the port 8080 which is default but is different from --ui-port
Flow URL: <http://localhost:8080/default/flow/ebf8a573-403a-4928-96ea-098dd1d5ca93>
└── ID: 42af3ef7-918e-4f03-ae5b-57a881fc87ec
└── Project: tutorial
The wget for the port 2222 outputs some html files, but <http://localhost:2222>
shows nothing.
wget <http://localhost:2222/default/flow/ebf8a573-403a-4928-96ea-098dd1d5ca93>
Kevin Kho
04/08/2022, 5:59 AM<http://localhost:8080>
or <http://localhost:2222>
to work