• Me
• Publications
• Teaching
• Software
• Blog

What to do with the results of upstream pipes

In the pipes library, the type of the composition operator is

(>+>) :: Pipe m a b r -> Pipe m b c r -> Pipe m a c r

If you look closely, then you will notice that all three pipes have result type r. How does this work? Simple: whichever pipe stops first provides the final result.

In my opinion this is wrong. The upstream pipe produces values, and the downstream pipe does something with them. The downstream pipe is the one that leads the computation, by pulling results from the upstream pipe. It is therefore always the downstream pipe that should provide the result. So, in the pipification of conduit, the proposed type for composition is instead

(>+>) :: Pipe m a b () -> Pipe m b c r -> Pipe m a c r

This makes it clear that the result of the first pipe is not used, the result of the composition always has to come from downstream. But now the result of the first pipe would be discarded completely.

Another, more general, solution is to communicate the result of the first pipe to the second one. That would give the await function in the downstream pipe the type

await :: Pipe m a b (Either r1 a)

where r₁ is the result of the upstream pipe. Of course that r₁ type needs to come from somewhere. So Pipe would need another type argument

data Pipe m streamin streamout finalin finalout

giving await the type

await :: Pipe m a b x (Either x a)

Composition becomes

(>+>) :: Pipe m a b x y -> Pipe m b c y z -> Pipe m a c x z

I think this makes Pipe into a category over pairs of Haskell types. I was tempted to call this a bicategory, in analogy with bifunctor, but that term apparently means something else.

Note that this article is just about a quick idea I had. I am not saying that this is the best way to do things. In fact, I am not even sure if propagating result values in this way actually helps solve any real world problems.

Comments

Sjoerd VisscherDate: 2012-04-05T14:55Zx

One obvious category of pairs of Haskell types is the product category. Then the arrows are also pairs of arrows. Maybe a pipe can be a tuple of yield/await/effect on the left and early-close/done on the right?

Twan van LaarhovenDate: 2012-04-05T16:08Zx

Sjoerd: if you use a product category, then it becomes impossible to have pipes that close early based on awaited input. If you ignore effects, then pipes are functions between lists with a value at the end.

data Stream a r = Done r | More a (Stream a r)

So Pipe a b x y ≈ (Stream a x -> Stream b y).

Sjoerd VisscherDate: 2012-04-05T16:51Zx

Ok. But, should pipes know how to convert x's into y's? Or do we have Pipe a b x y ≈ (Stream a x -> Stream b (Either x y))?

Twan van LaarhovenDate: 2012-04-05T17:00Zx

I think that the pipe itself should know how to convert x into y. That's also how it works right now in conduit, with a=(). You would have, for instance

await :: Pipe a b x (Either a x)
await (Done x) = Done (Right x)
await (More a _) = Done (Left a)

The user of await would have to know what to do in the Right case.

The last case also illustrates that this implementation is not good enough if you want to use Pipe as a monad. Then you would also need to keep track of the left-over input somehow.

In pipes-core, the await function is made more convenient by adding an error monad on top of pipes, so await would throw an exception when there is no more input. You could easily do that yourself with ErrorT x (Pipe a b x) y.

Reply