Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences
- Introduction
- Related work
- Dataset
- Learning the style space
- Generating outfits
- Visualizing the style space
- Evaluation
- Conclusion
(ICCV 2015) Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences
Paper: http://cseweb.ucsd.edu/~jmcauley/pdfs/iccv15.pdf
Supplement: https://goo.gl/OM1rAL
Code: https://vision.cornell.edu/se3/projects/clothing-style/
Introduction
However, these approaches require significant domain knowledge and do not generalize well to the introduction of new subcategories. Further, they require large datasets with fine grained category labels, which are difficult to collect.
learning a feature transformation from the images of the items to a latent space, which we call style space.
The challenge of this problem is that the bundle of objects come from visually distinct categories.
First, the input data comprises item images, category labels and links between items, describing co-occurrences.
Then, to learn style across categories, we strategically sample training examples from the input data such that pairs of items are co-occurring heterogeneous dyads, i.e., the two items belong to different categories and frequently co-occur. Subsequently, we use Siamese CNNs to learn a feature transformation from the image space to the latent style space.
Finally, we generate structured bundles of compatible items by querying the learned latent space and retrieving the nearest neighbors from each category to the query item.
Contributions:
- combines Siamese CNNs with co-occurrence information as well as category labels
- allows learning compatibility across categories
- a robust nearest neighbor retrieval method for datasets with strong label noise
- a user study to understand how users think about style and compatibility
Related work
Dataset
requires positive and negative examples of clothing pairs.
Most of the images are iconic with a white background. However, some products are shown in full body pictures.
