Using Flow API to Compose Your Jina Workflow¶
In this section, you will get to know how to construct the flow using different approaches.
Table of Contents
Using Flow API to Compose Your Jina Workflow
Feature description and expected outcome
-
-
Feature description and expected outcome linking.
Before you start¶
Make sure you install latest version of Jina via Installation.
Implementation¶
Use Flow API in Python¶
Create a Flow¶
To create a new Flow:
from jina.flow import Flow f = Flow()
Flow() accepts some arguments. See
jina flow --help or check here for details.(), syntax of YAML file can be found here:.examples.mwu_encoder:latest') .add(name='p3'))
This will run
p2 in a Docker container equipped with the image
jinaai/hub.examples.mwu_encoder:latest. More information on using containerized Pod can be found here. here.
Add a Remote Containerized Pod into the Flow¶
A very useful pattern is to combine the above two features together:
f = (Flow().add(name='p1') .add(name='p2', host='192.168.0.100', port_expose=53100, image='jinaai/hub.examples.mwu_encoder here.
Parallelize the Steps¶
By default, if you keep adding
.add() to a
Flow, it will create a sequential workflow chain.(name='p1') .add(name='p2') .add(name='p3', needs='p1') .join(['p2', 'p3'])).
Test Connectivity with Dry Run¶
You can test the whole workflow with
pass. For example:
with f: pass
Iterate over Pods in the Flow¶
You can iterate the Pods in a Flow like you would a list. For example:
f = (Flow().add(name='p1') .add(name='p2') .add(name='p3', needs='p1') .join(['p2', 'p3'])) for p in f.build(): print(f'name: {p[0]} in: {str(p[1].args.socket_in)} out: {str(p[1], on_done=print)
input_fnis an
Iterator[bytes], each of which corresponds to the representation of a Document with bytes.
on_doneis()
WARNING: don’t use a while loop to do the waiting, it is extremely inefficient:
with f: while True: # <- dont do that pass # <- dont do that
Use Flow API in YAML¶
You can also write a Flow in YAML:
!Flow version: '1.0' with: restful: true pods: - name: pod0 # notice the change here, name is now an attribute method: add # by default method is always add, available: add, needs, inspect uses: _pass needs: gateway - name: pod1 # notice the change here, name is now an attribute method: add # by default method is always add, available: add, needs, inspect uses: _pass needs: gateway - method: inspect # add an inspect node on pod1 - method: needs # let's try something new in Flow YAML v1: needs needs: [pod1, pod0]
You can use enviroment variables with
$ in YAML. More information on the Flow YAML Schema can be found here.