Image Search

Overview

Image search lets users find products by uploading a photo. Refine analyzes the image’s visual features and returns products that look similar. Perfect for “shop the look” experiences, visual discovery, and reverse image search.

Basic Usage

import { Refine } from '@refine-ai/sdk';

const refine = new Refine({
  apiKey: process.env.REFINE_API_KEY,
  organizationId: 'org_abc123',
  catalogId: 'cat_xyz789'
});

// With a File object (from file input)
const fileInput = document.querySelector('input[type="file"]');
const file = fileInput.files[0];

const results = await refine.search.image({
  image: file,
  topK: 12
});

Image Input Formats

The SDK accepts three image formats:

File Object
Blob
Base64 String

// From file input
const input = document.querySelector<HTMLInputElement>('#image-upload');
input.addEventListener('change', async (e) => {
  const file = input.files?.[0];
  if (file) {
    const results = await refine.search.image({
      image: file,
      topK: 12
    });
  }
});

// From canvas or fetch
const canvas = document.querySelector('canvas');
canvas.toBlob(async (blob) => {
  const results = await refine.search.image({
    image: blob,
    topK: 12
  });
});

// From URL
const response = await fetch('https://example.com/image.jpg');
const blob = await response.blob();
const results = await refine.search.image({
  image: blob,
  topK: 12
});

// From data URL
const dataUrl = 'data:image/jpeg;base64,/9j/4AAQSkZJRg...';
const base64 = dataUrl.split(',')[1];

const results = await refine.search.image({
  image: base64,
  topK: 12
});

Parameters

File | Blob | string

required

The image to search with. Accepts File objects, Blobs, or base64-encoded strings.

number

required

Number of results to return. Recommended: 8-24 for image search.

Filter[]

Array of filters to apply. See Filters.

With Filters

Combine visual search with filters to narrow results:

const results = await refine.search.image({
  image: uploadedFile,
  topK: 20,
  filters: [
    { field: 'price', operator: 'lte', value: 150 },
    { field: 'metadata.category', operator: 'eq', value: 'dresses' },
    { field: 'stock', operator: 'gt', value: 0 }
  ]
});

Response

The response structure is identical to text search:

interface SearchResponse {
  results: SearchResultItem[];
  filterOptions?: FilterOption[];
  serveId: string;
  totalResults: number;
}

interface SearchResultItem {
  productId: string;
  title: string;
  price: number;
  imageUrl: string;
  metadata: Record<string, any>;
  score: number;  // Visual similarity score
}

Tracking Image Search

Track image search results the same way as text search:

const results = await refine.search.image({ image: file, topK: 12 });

const searchContext = refine.events.trackItemsServed({
  surface: 'search_results',
  source: 'image-search',
  itemIds: results.results.map(r => r.productId),
  totalResults: results.totalResults
});

// Track interactions
searchContext.trackClick(productId, position);

Complete Implementation

Here’s a full drag-and-drop image search component:

import { Refine } from '@refine-ai/sdk';

const refine = new Refine({
  apiKey: process.env.REFINE_API_KEY,
  organizationId: 'org_abc123',
  catalogId: 'cat_xyz789'
});

class ImageSearchUploader {
  private dropzone: HTMLElement;
  private fileInput: HTMLInputElement;
  private resultsContainer: HTMLElement;
  private searchContext: any;

  constructor() {
    this.dropzone = document.getElementById('dropzone')!;
    this.fileInput = document.getElementById('file-input') as HTMLInputElement;
    this.resultsContainer = document.getElementById('results')!;
    
    this.setupEventListeners();
  }

  private setupEventListeners() {
    // Drag and drop
    this.dropzone.addEventListener('dragover', (e) => {
      e.preventDefault();
      this.dropzone.classList.add('drag-over');
    });

    this.dropzone.addEventListener('dragleave', () => {
      this.dropzone.classList.remove('drag-over');
    });

    this.dropzone.addEventListener('drop', async (e) => {
      e.preventDefault();
      this.dropzone.classList.remove('drag-over');
      const file = e.dataTransfer?.files[0];
      if (file && file.type.startsWith('image/')) {
        await this.search(file);
      }
    });

    // File input
    this.fileInput.addEventListener('change', async () => {
      const file = this.fileInput.files?.[0];
      if (file) {
        await this.search(file);
      }
    });
  }

  private async search(file: File) {
    this.resultsContainer.innerHTML = '<p>Searching...</p>';

    try {
      const results = await refine.search.image({
        image: file,
        topK: 16
      });

      // Track the search
      this.searchContext = refine.events.trackItemsServed({
        surface: 'search_results',
        source: 'image-search',
        itemIds: results.results.map(r => r.productId),
        totalResults: results.totalResults
      });

      this.renderResults(results.results);
    } catch (error) {
      this.resultsContainer.innerHTML = '<p>Search failed. Please try again.</p>';
      console.error('Image search error:', error);
    }
  }

  private renderResults(products: SearchResultItem[]) {
    this.resultsContainer.innerHTML = products.map((product, index) => `
      <div class="product" data-id="${product.productId}" data-position="${index}">
        <img src="${product.imageUrl}" alt="${product.title}" />
        <h3>${product.title}</h3>
        <p>$${product.price.toFixed(2)}</p>
        <p class="score">Match: ${(product.score * 100).toFixed(0)}%</p>
      </div>
    `).join('');

    // Attach click handlers
    this.resultsContainer.querySelectorAll('.product').forEach(el => {
      el.addEventListener('click', () => {
        const productId = (el as HTMLElement).dataset.id!;
        const position = parseInt((el as HTMLElement).dataset.position!);
        this.searchContext?.trackClick(productId, position);
      });
    });
  }
}

// Initialize
new ImageSearchUploader();

Image Requirements

For best results:

Requirement	Recommendation
Format	JPEG, PNG, WebP
Size	Max 10MB
Dimensions	At least 200x200px
Quality	Clear, well-lit images
Content	Single product focus works best

Large images are resized server-side. For faster uploads, resize images to ~800px on the longest edge before sending.

Use Cases

Shop the Look

Let users upload photos of outfits they like and find similar items in your catalog.

Visual Discovery

Add a camera icon to your search bar for users who prefer visual browsing.

Screenshot Search

Users can screenshot products from social media and find them in your store.

Competitor Matching

Find similar products to items users found elsewhere (use responsibly).

Error Handling

import { RefineValidationError } from '@refine-ai/sdk';

try {
  const results = await refine.search.image({ image: file, topK: 12 });
} catch (error) {
  if (error instanceof RefineValidationError) {
    if (error.message.includes('file size')) {
      alert('Image too large. Please use an image under 10MB.');
    } else if (error.message.includes('format')) {
      alert('Unsupported format. Please use JPEG, PNG, or WebP.');
    }
  }
}

Getting Started

SDK

Search

Recommendations

Product Listing Pages

Event Tracking

Identity

Plugins

Advanced

Overview

Basic Usage

Image Input Formats

Parameters

With Filters

Response

Tracking Image Search

Complete Implementation

Image Requirements

Use Cases

Error Handling

Next Steps

Text Search

Filters

​Overview

​Basic Usage

​Image Input Formats

​Parameters

​With Filters

​Response

​Tracking Image Search

​Complete Implementation

​Image Requirements

​Use Cases

​Error Handling

​Next Steps

Text Search

Filters

Overview

Basic Usage

Image Input Formats

Parameters

With Filters

Response

Tracking Image Search

Complete Implementation

Image Requirements

Use Cases

Error Handling

Next Steps