Skip to main content

Overview

Image search lets users find products by uploading a photo. Refine analyzes the image’s visual features and returns products that look similar. Perfect for “shop the look” experiences, visual discovery, and reverse image search.

Basic Usage

import { Refine } from '@refine-ai/sdk';

const refine = new Refine({
  apiKey: process.env.REFINE_API_KEY,
  organizationId: 'org_abc123',
  catalogId: 'cat_xyz789'
});

// With a File object (from file input)
const fileInput = document.querySelector('input[type="file"]');
const file = fileInput.files[0];

const results = await refine.search.image({
  image: file,
  topK: 12
});

Image Input Formats

The SDK accepts three image formats:
  • File Object
  • Blob
  • Base64 String
// From file input
const input = document.querySelector<HTMLInputElement>('#image-upload');
input.addEventListener('change', async (e) => {
  const file = input.files?.[0];
  if (file) {
    const results = await refine.search.image({
      image: file,
      topK: 12
    });
  }
});

Parameters

image
File | Blob | string
required
The image to search with. Accepts File objects, Blobs, or base64-encoded strings.
topK
number
required
Number of results to return. Recommended: 8-24 for image search.
filters
Filter[]
Array of filters to apply. See Filters.

With Filters

Combine visual search with filters to narrow results:
const results = await refine.search.image({
  image: uploadedFile,
  topK: 20,
  filters: [
    { field: 'price', operator: 'lte', value: 150 },
    { field: 'metadata.category', operator: 'eq', value: 'dresses' },
    { field: 'stock', operator: 'gt', value: 0 }
  ]
});

Response

The response structure is identical to text search:
interface SearchResponse {
  results: SearchResultItem[];
  filterOptions?: FilterOption[];
  serveId: string;
  totalResults: number;
}

interface SearchResultItem {
  productId: string;
  title: string;
  price: number;
  imageUrl: string;
  metadata: Record<string, any>;
  score: number;  // Visual similarity score
}
Track image search results the same way as text search:
const results = await refine.search.image({ image: file, topK: 12 });

const searchContext = refine.events.trackItemsServed({
  surface: 'search_results',
  source: 'image-search',
  itemIds: results.results.map(r => r.productId),
  totalResults: results.totalResults
});

// Track interactions
searchContext.trackClick(productId, position);

Complete Implementation

Here’s a full drag-and-drop image search component:
import { Refine } from '@refine-ai/sdk';

const refine = new Refine({
  apiKey: process.env.REFINE_API_KEY,
  organizationId: 'org_abc123',
  catalogId: 'cat_xyz789'
});

class ImageSearchUploader {
  private dropzone: HTMLElement;
  private fileInput: HTMLInputElement;
  private resultsContainer: HTMLElement;
  private searchContext: any;

  constructor() {
    this.dropzone = document.getElementById('dropzone')!;
    this.fileInput = document.getElementById('file-input') as HTMLInputElement;
    this.resultsContainer = document.getElementById('results')!;
    
    this.setupEventListeners();
  }

  private setupEventListeners() {
    // Drag and drop
    this.dropzone.addEventListener('dragover', (e) => {
      e.preventDefault();
      this.dropzone.classList.add('drag-over');
    });

    this.dropzone.addEventListener('dragleave', () => {
      this.dropzone.classList.remove('drag-over');
    });

    this.dropzone.addEventListener('drop', async (e) => {
      e.preventDefault();
      this.dropzone.classList.remove('drag-over');
      const file = e.dataTransfer?.files[0];
      if (file && file.type.startsWith('image/')) {
        await this.search(file);
      }
    });

    // File input
    this.fileInput.addEventListener('change', async () => {
      const file = this.fileInput.files?.[0];
      if (file) {
        await this.search(file);
      }
    });
  }

  private async search(file: File) {
    this.resultsContainer.innerHTML = '<p>Searching...</p>';

    try {
      const results = await refine.search.image({
        image: file,
        topK: 16
      });

      // Track the search
      this.searchContext = refine.events.trackItemsServed({
        surface: 'search_results',
        source: 'image-search',
        itemIds: results.results.map(r => r.productId),
        totalResults: results.totalResults
      });

      this.renderResults(results.results);
    } catch (error) {
      this.resultsContainer.innerHTML = '<p>Search failed. Please try again.</p>';
      console.error('Image search error:', error);
    }
  }

  private renderResults(products: SearchResultItem[]) {
    this.resultsContainer.innerHTML = products.map((product, index) => `
      <div class="product" data-id="${product.productId}" data-position="${index}">
        <img src="${product.imageUrl}" alt="${product.title}" />
        <h3>${product.title}</h3>
        <p>$${product.price.toFixed(2)}</p>
        <p class="score">Match: ${(product.score * 100).toFixed(0)}%</p>
      </div>
    `).join('');

    // Attach click handlers
    this.resultsContainer.querySelectorAll('.product').forEach(el => {
      el.addEventListener('click', () => {
        const productId = (el as HTMLElement).dataset.id!;
        const position = parseInt((el as HTMLElement).dataset.position!);
        this.searchContext?.trackClick(productId, position);
      });
    });
  }
}

// Initialize
new ImageSearchUploader();

Image Requirements

For best results:
RequirementRecommendation
FormatJPEG, PNG, WebP
SizeMax 10MB
DimensionsAt least 200x200px
QualityClear, well-lit images
ContentSingle product focus works best
Large images are resized server-side. For faster uploads, resize images to ~800px on the longest edge before sending.

Use Cases

Let users upload photos of outfits they like and find similar items in your catalog.
Add a camera icon to your search bar for users who prefer visual browsing.
Find similar products to items users found elsewhere (use responsibly).

Error Handling

import { RefineValidationError } from '@refine-ai/sdk';

try {
  const results = await refine.search.image({ image: file, topK: 12 });
} catch (error) {
  if (error instanceof RefineValidationError) {
    if (error.message.includes('file size')) {
      alert('Image too large. Please use an image under 10MB.');
    } else if (error.message.includes('format')) {
      alert('Unsupported format. Please use JPEG, PNG, or WebP.');
    }
  }
}

Next Steps