Zapier's Deviation from AI Tool Adoption Trends in AI Hype Cycle and Potential Comeback

Diverging trends suggest automation tools faced challenges during AI hype cycle - will trend reverse as hype subsides?

Sep 04, 2024

Zapier's public developer activity declined (-13.1% y/y) until last month, while AI-related APIs have experienced steady growth (+12.0% y/y) during same timeframe.
Zapier's recent spike may indicate strategic adaptation or solution to AI trends - highest correlation to UIPath during a period of aggressive free tool messaging, but correlation doesn’t equal causation either way.
Caveat on public developer activity, so not accounting for private trends (which could be substantially different).

Question this quick analysis answers:

Did AI hype-infused solutions to workflow automation effect trends with Zapier’s workflow automation solutions, and could that be shaking out differently at inflection point in hype cycle?

Let's start by importing the necessary libraries and loading our data (see my previous blog post for public development trend query out of GCP). Note this code is on my github repo in the form of a notebook.

# imports
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from scipy import stats
import numpy as np

# Load the data - in this case sourced from same query over weekend
data = pd.read_csv('ff.csv')

Long table format, so transformations are called for

# Convert 'month' to datetime
data['month'] = pd.to_datetime(data['month'])

# Filter out September 2024 - incomplete month
data = data[data['month'] < '2024-09-01']

# Filter data for the complete years (2023 and 2024)
data = data[data['month'].dt.year.isin([2023, 2024])]

# Separate Zapier data
zapier_data = data[data['keyword_category'] == 'zapier'].set_index('month')

# Aggregate all other categories as 'AI-related APIs'
ai_apis_data = data[data['keyword_category'] != 'zapier'].groupby('month')['new_repo_count'].sum().reset_index()
ai_apis_data = ai_apis_data.set_index('month')

# Calculate 7-day rolling average for smoothing
zapier_data['rolling_avg'] = zapier_data['new_repo_count'].rolling(window=7).mean()
ai_apis_data['rolling_avg'] = ai_apis_data['new_repo_count'].rolling(window=7).mean()

Zapier data I’d queried is so small (!) so the month over month variation isn’t going to lend to anything stat sig, by month, but in aggregate it’s likely going to help support a hypothesis

# Calculate 95% confidence intervals
def calculate_ci(data):
    confidence = 0.95
    degrees_of_freedom = len(data) - 1
    sample_mean = np.mean(data)
    sample_standard_error = stats.sem(data)

    ci = stats.t.interval(confidence=confidence,
                          df=degrees_of_freedom,
                          loc=sample_mean,
                          scale=sample_standard_error)
    return ci

zapier_ci = calculate_ci(zapier_data['new_repo_count'])
ai_apis_ci = calculate_ci(ai_apis_data['new_repo_count'])

And just since I mentioned it, quick aggregate to compare Y/Y

# Calculate Y/Y growth for Jan-July period
def calculate_yoy_growth(data, year1, year2):
    jan_jul_year1 = data[(data.index.year == year1) & (data.index.month.isin(range(1, 8)))]['new_repo_count'].sum()
    jan_jul_year2 = data[(data.index.year == year2) & (data.index.month.isin(range(1, 8)))]['new_repo_count'].sum()
    return (jan_jul_year2 - jan_jul_year1) / jan_jul_year1 * 100

zapier_yoy = calculate_yoy_growth(zapier_data, 2023, 2024)
ai_apis_yoy = calculate_yoy_growth(ai_apis_data, 2023, 2024)

Plotting this result, it’s easy to see the divergence during the AI hype cycle timeframe

# Create the plot
fig, ax1 = plt.subplots(figsize=(12, 7))

# Plot Zapier data on the left y-axis
ax1.plot(zapier_data.index, zapier_data['rolling_avg'], color='blue', label='Zapier')

# Set up the right y-axis for AI-related APIs
ax2 = ax1.twinx()
ax2.plot(ai_apis_data.index, ai_apis_data['rolling_avg'], color='red', label='AI-related APIs')

# Customize the plot
ax1.set_xlabel('Date')
ax1.set_ylabel('New Repo Count (Zapier)', color='blue')
ax2.set_ylabel('New Repo Count (AI-related APIs)', color='red')
ax1.tick_params(axis='y', labelcolor='blue')
ax2.tick_params(axis='y', labelcolor='red')

# Add legend
lines1, labels1 = ax1.get_legend_handles_labels()
lines2, labels2 = ax2.get_legend_handles_labels()
ax1.legend(lines1 + lines2, labels1 + labels2, loc='upper left')

# Set title and subtitle
plt.title("Public API Usage Trends Y/Y", fontsize=16, pad=20)
plt.figtext(0.7, 0.80, f"Zapier Y/Y Growth: {zapier_yoy:.1f}%, AI-related APIs Y/Y Growth: {ai_apis_yoy:.1f}%\n"
                       f"(Based on Jan-Jul trends) * not statistically significant at 95% CI",
            fontsize=10, ha='center')

# Adjust layout
plt.tight_layout()
plt.subplots_adjust(top=0.85)  # Adjust top margin to accommodate subtitle

# Show the plot
plt.show()

Does this correlate to any specific packages…? The plot below shows UIPath correlation - while this doesn’t equal causation obv, messaging from this company became aggressive in recent months towards the scholastic communities (free tools) - C3.ai data is dirty but also worth noting some correlation to Oracle AI and Google Vertex tools.

# Create a pivot table with months as index and keyword categories as columns
pivot_data = data.pivot_table(values='new_repo_count', index='month', columns='keyword_category', aggfunc='sum')

# Calculate correlation between Zapier and other categories
correlations = pivot_data.corrwith(pivot_data['zapier']).sort_values(ascending=False)

# Remove Zapier's self-correlation and any NaN values
correlations = correlations.drop('zapier').dropna()

# Get the top 5 correlated categories
top_5_correlations = correlations.head(5)

print("Top 5 dimensions correlated with Zapier:")
for category, correlation in top_5_correlations.items():
    print(f"{category}: {correlation:.4f}")

# Plot the correlation results for top 5
plt.figure(figsize=(12, 6))
top_5_correlations.plot(kind='bar')
plt.title("Top 5 Correlations (again, sans CI): Developer Usage of Zapier vs Other Categories")
plt.xlabel("Categories")
plt.ylabel("Correlation Coefficient")
plt.xticks(rotation=45, ha='right')
plt.tight_layout()
plt.show()

Summary:

1. Shift in Developer Focus in Past Year:

The declining trend for Zapier activity could indicate a shift in developer focus away from traditional automation platforms towards AI-centric technologies that were attempting to accomplish similar goals.

2. Recent Upturn for Zapier

The sharp increase in Zapier's trend recently could be attributed to:

Introduction of AI-related Features: Zapier may have introduced new AI-centric capabilities or integrations, sparking renewed interest among developers.
AI hype may not have automated what developers were trying to do: There is no data to suggest this, since AI APIs are still increasing in usage.
Synergy with AI Technologies: The rise could reflect Zapier's efforts to incorporate AI into its platform, possibly something involving free tools or UIPath, and also potentially offering new ways for developers to leverage both automation and AI capabilities together.

It's important to note that while these trends provide insights into developer interests and industry directions, they may not capture the full complexity of the API ecosystem. Factors such as changes in Zapier's business strategy, shifts in the broader tech landscape, and the emergence of new competitors could also play roles in shaping these trends (in theory).