Skip to main content

​
Overview

The ctx.utils object provides utility functions for common data transformations and normalizations. These helpers make it easier to work with URLs, strings, and other data types in your formulas.

​
Available Utilities

​
ensureHttp()

Ensure a URL has a protocol (http or https). If the URL already has a protocol, it is returned unchanged. Otherwise, https:// is prepended. 
ctx.utils.ensureHttp(url: string): string
Parameters:
  • url - The URL to normalize
Returns: The URL with a protocol Example: 
// Add protocol to URLs without one
const url1 = ctx.utils.ensureHttp("example.com");
// Returns: "https://example.com"

const url2 = ctx.utils.ensureHttp("http://example.com");
// Returns: "http://example.com" (unchanged)

const url3 = ctx.utils.ensureHttp("https://example.com");
// Returns: "https://example.com" (unchanged)
Common Use Case: 
// Normalize user-provided URLs before scraping
const rawUrl = ctx.thisRow.get("website");
const normalizedUrl = ctx.utils.ensureHttp(rawUrl);

const content = await services.scrape.website({
  url: normalizedUrl,
});

return content.markdown;

​
normalizeUrl()

Normalize a URL by removing the protocol, www., paths, and query parameters. This is useful for deduplication and comparison. 
ctx.utils.normalizeUrl(url: string): string
Parameters:
  • url - The URL to normalize
Returns: The normalized URL (domain only, no protocol or www) Example: 
// Normalize various URL formats to the same domain
const url1 = ctx.utils.normalizeUrl("https://www.example.com/about?ref=home");
// Returns: "example.com"

const url2 = ctx.utils.normalizeUrl("http://example.com/contact");
// Returns: "example.com"

const url3 = ctx.utils.normalizeUrl("www.example.com");
// Returns: "example.com"

// All three return the same normalized domain
Common Use Case: 
// Deduplicate companies by domain
const website = ctx.thisRow.get("website");
const normalizedDomain = ctx.utils.normalizeUrl(website);

// Check if this domain already exists
const existingCompany = await ctx.getRowByValue(
  "normalized_domain",
  normalizedDomain
);

if (existingCompany) {
  return "duplicate";
}

return normalizedDomain;

​
Best Practices

URL Normalization Tips:
  1. Always normalize for deduplication: Use normalizeUrl() to create a consistent domain format for matching records
  2. Ensure protocol before API calls: Use ensureHttp() before passing URLs to services that require full URLs
  3. Handle edge cases: Check for null/empty values before normalizing