Hello, I am trying to perform the following transformation, ideally using existing Caffe2 operators:
Inputs:
  boxes.shape=[nb_boxes, 6]
    where each row boxes[i, :] -> [x1, y1, x2, y2, float confidence, int category_index].
    notably, boxes is arranged sequentially by image
  batch_splits.shape=[nb_images]
    Denotes nb of boxes for each image in the batch.
    Example: if batch_splits = [42, 3], then the first 42 boxes in `boxes` belong to the first image, and the next 3 boxes belong to the second image.
Output:
  boxes_by_batch.shape=[nb_images, nb_boxes_max, 6]
    A rearrangement of boxes, but in a familiar batch-style format, eg boxes_by_batch[0, :, :] contains boxes for image 0, boxes_by_batch[1, :, :] contains boxes for image 1, etc. 
    This is zero-filled: important, because each image may have different nb of detected boxes. The second dimension nb_boxes_max is the largest nb of boxes in an image in the batch.

Here is a specific example + code that better illustrates what I’m trying to do:
import numpy as np
from caffe2.python import workspace

# [1/3] Initialize input data
boxes = np.array([
    [10, 20, 100, 200, 0.75, 1],
    [20, 30, 50, 70, 0.95, 3],
    [80, 100, 25, 25, 0.5, 2]
], dtype=np.float32)
# batch_splits indicates that first two boxes belong to img0, and third box
# belongs to img1
batch_splits = np.array([2, 1], dtype=np.float32)
workspace.FeedBlob("boxes", boxes)
workspace.FeedBlob("batch_splits", batch_splits)

# [2/3] Add op(s) that emit boxes_by_batch
ops = [] # TODO: fill me!
workspace.RunOperatorsOnce(ops)

# [3/3] Fetch boxes_by_batch, compare to desired output
boxes_by_batch = workspace.FetchBlob("boxes_by_batch")
boxes_by_batch_desired = np.array([
    [
        [10, 20, 100, 200, 0.75, 1],
        [20, 30, 50, 70, 0.95, 3]
    ],
    [
        [80, 100, 25, 25, 0.5, 2],
        [0, 0, 0, 0, 0, 0]
    ]
])
print("same? {}".format(np.allclose(boxes_by_batch, boxes_by_batch_desired)))

Context: I’m trying to serve a Detectron object detection model (FRCNN+FRN) with batch inference enabled. The Detectron model emits detected boxes in the boxes, batch_splits format, but for my usecase it’d be easier if the network instead emitted the detected boxes in boxes_by_batch format.
Ideally, I’d like to use existing Caffe2 operators so that I don’t have to write a custom C++ operator (and deal with linking it to our prod env, etc).
Thank you!