Rings a bell from a discussion a few years ago. Concensus then was that it wasn't the best of idea, possibly not allowed by affiliates, and technically full of issues.
If all the items are one affiliate then it's viable - the way that amazon affiliate store carts work. But for multiple affiliate stores then it wouldn't work. Where would you send the shopper to?
There was also the issue raised of if you put a cart on your site are you implying that your are responsible for taking payment.
I've developed affiliate product amalgamation sites and this is a non starter imo.
There is one site that I know of that does something along that line, but has multiple levels of products in the site. Some pure affiliate products, some custom merchants feeding products, some product fullfillment/dropship. This is a bespoke example though: furnish.co.uk