Every enterprise analytics platform sooner or later faces the same challenge. More content is generated than removed. More dashboards are created. More data sources are uploaded. More workflows are launched. Projects grow. Teams evolve. People move on. But nothing much actually gets removed.

In small-scale settings, this poses little operational problem. In enterprise-scale environments, however, things become challenging quite quickly. Ineffective use of disk space becomes an issue. Scheduled refreshes continue running for weeks or months after being no longer needed. Backups continue consuming increasing amounts of resources. Performance gradually worsens.

Cleaning up seems the obvious way out. Automated deletion of unused content becomes the tempting solution.

Except...

What if something important ends up being accidentally deleted? Thousands of users will be affected, and their jobs will become a lot harder overnight.

That was the problem our engineering team had to face while working with a large analytics platform featuring 14 servers and hundreds of thousands of users. Neither manual nor fully automated content cleanups were viable options for us. So, the solution we had to design combined manual validation and automation to ensure safety while achieving operational benefits.

The Challenge of Identifying Stale Content

Most organizations tackle content cleanup through one of two approaches.

The first involves manual reviews. Admins check which content is no longer being used and remove it from the platform. This approach works perfectly well in small-scale environments. It's totally impractical in enterprise-scale settings where the number of assets is too large for humans to process manually.

The second approach relies on automated deletion based on inactivity criteria. As long as content remains unused for a specified period, it gets removed automatically.

Unfortunately, different kinds of content behave very differently. Some files get used every day. Others get accessed only once or twice per month. Some are only needed quarterly or annually. Others may appear inactive but still participate in scheduled refreshes or business processes.

As we explored both approaches, we concluded that neither solved the problem effectively. Manual review was operationally impossible. Automated deletion introduced too much risk.

The challenge wasn't identifying stale content. The challenge was doing so safely.

Designing a System for Safe Failure

The principle that ultimately guided the architecture was simple:

Don't delete anything you can't restore.

This requirement shaped every decision that followed.

Traditional cleanup automation typically looks like this:

Identify Stale Content
          ↓
Delete Content

Our system followed a different path:

By separating quarantine from deletion, we fundamentally changed the system's failure modes. If content was incorrectly classified, it could still be recovered. If a stakeholder realized an asset was still needed, intervention remained possible. The system was intentionally engineered so that its default failure mode was inconvenience rather than data loss.

Building the Content Discovery Engine

The first major component of the platform was the content discovery engine.

Its objective was straightforward:

Identify content that had not been used within a specific period.

At first glance, this sounds simple.

Most platforms expose metadata such as:

• Last modified date
• Last accessed date
• Ownership information
• Scheduled refresh information

def fetch_all_assets(connections):
    """Iterate server connections, join asset + ownership data."""
    all_assets = []

    query = """
        SELECT a.asset_id, a.last_modified, a.last_accessed,
               a.refresh_success_rate, u.user_id AS owner_id, u.is_active AS owner_active
        FROM assets a
        LEFT JOIN users u ON a.owner_id = u.user_id
        WHERE a.deleted_at IS NULL
    """

    for conn_info in connections:
        try:
            with psycopg2.connect(**conn_info) as conn, conn.cursor() as cur:
                cur.execute(query)
                for row in cur.fetchall():
                    all_assets.append(dict(zip(
                        ["asset_id", "last_modified", "last_accessed",
                         "refresh_success_rate", "owner_id", "owner_active"],
                        row
                    )))
        except Exception as e:
            logger.error(f"Skipping {conn_info['host']}: {e}")  # one bad server shouldn't halt the run

    return all_assets

However, reality proved more complicated. Certain content types generated incomplete usage data. Embedded dashboards sometimes produced no meaningful activity signals. Some assets were accessed indirectly through external applications. Others participated in automated processes without receiving direct user traffic.

As a result, we had to move beyond simple metadata checks and evaluate multiple signals simultaneously.

The platform incorporated factors such as:

• Access frequency
• Refresh activity
• Embedded usage patterns
• Ownership status

def staleness_score(asset):
    score = 0
    score += min(asset.days_since_last_access / 90, 1.0) * 0.35
    score += (1 - asset.refresh_success_rate) * 0.25
    score += (0 if asset.embedded_usage_detected else 1) * 0.20
    score += (1 if asset.owner_inactive else 0) * 0.20
    return score  # 0 = active, 1 = strong stale candidate

Rather than relying on a single field, content activity became a composite score built from multiple indicators.

Why a Universal Retention Policy Fails

Another challenge emerged early in development. Initially, we considered applying a universal inactivity threshold to all content. That quickly proved problematic.

Different environments operate under different business rhythms. Development environments tend to have short lifecycles. Financial reporting environments often follow quarterly cycles. Compliance-related assets may only be accessed once per year. Applying the same threshold everywhere creates unnecessary risk.

To address this, we introduced configurable retention policies. Retention could vary based on:

• Environment type
• Project classification
• Known usage patterns
• Business requirements

Environment type	Inactivity threshold before quarantine	Grace period
Development/sandbox	~30 days	3 days
Standard reporting	~90 days	14 days
Financial/quarterly cycles	~120 days	30 days
Compliance/audit	~365 days	45 days

This flexibility allowed lifecycle management to reflect operational reality rather than arbitrary platform rules.

The Importance of the Quarantine Layer

Quarantine became the most important stage of the entire workflow.<

Every enterprise analytics platform sooner or later faces the same challenge. More content is generated than removed. More dashboards are created. More data sources are uploaded. More workflows are launched. Projects grow. Teams evolve. People move on. But nothing much actually gets removed.

In small-scale settings, this poses little operational problem. In enterprise-scale environments, however, things become challenging quite quickly. Ineffective use of disk space becomes an issue. Scheduled refreshes continue running for weeks or months after being no longer needed. Backups continue consuming increasing amounts of resources. Performance gradually worsens.

Cleaning up seems the obvious way out. Automated deletion of unused content becomes the tempting solution.

Except...

What if something important ends up being accidentally deleted? Thousands of users will be affected, and their jobs will become a lot harder overnight.

That was the problem our engineering team had to face while working with a large analytics platform featuring 14 servers and hundreds of thousands of users. Neither manual nor fully automated content cleanups were viable options for us. So, the solution we had to design combined manual validation and automation to ensure safety while achieving operational benefits.

The Challenge of Identifying Stale Content

Most organizations tackle content cleanup through one of two approaches.

The first involves manual reviews. Admins check which content is no longer being used and remove it from the platform. This approach works perfectly well in small-scale environments. It's totally impractical in enterprise-scale settings where the number of assets is too large for humans to process manually.

The second approach relies on automated deletion based on inactivity criteria. As long as content remains unused for a specified period, it gets removed automatically.

Unfortunately, different kinds of content behave very differently. Some files get used every day. Others get accessed only once or twice per month. Some are only needed quarterly or annually. Others may appear inactive but still participate in scheduled refreshes or business processes.

As we explored both approaches, we concluded that neither solved the problem effectively. Manual review was operationally impossible. Automated deletion introduced too much risk.

The challenge wasn't identifying stale content. The challenge was doing so safely.

Designing a System for Safe Failure

The principle that ultimately guided the architecture was simple:

Don't delete anything you can't restore.

This requirement shaped every decision that followed.

Traditional cleanup automation typically looks like this:

Identify Stale Content
          ↓
Delete Content

Our system followed a different path:

By separating quarantine from deletion, we fundamentally changed the system's failure modes. If content was incorrectly classified, it could still be recovered. If a stakeholder realized an asset was still needed, intervention remained possible. The system was intentionally engineered so that its default failure mode was inconvenience rather than data loss.

Building the Content Discovery Engine

The first major component of the platform was the content discovery engine.

Its objective was straightforward:

Identify content that had not been used within a specific period.

At first glance, this sounds simple.

Most platforms expose metadata such as:

• Last modified date
• Last accessed date
• Ownership information
• Scheduled refresh information

def fetch_all_assets(connections):
    """Iterate server connections, join asset + ownership data."""
    all_assets = []

    query = """
        SELECT a.asset_id, a.last_modified, a.last_accessed,
               a.refresh_success_rate, u.user_id AS owner_id, u.is_active AS owner_active
        FROM assets a
        LEFT JOIN users u ON a.owner_id = u.user_id
        WHERE a.deleted_at IS NULL
    """

    for conn_info in connections:
        try:
            with psycopg2.connect(**conn_info) as conn, conn.cursor() as cur:
                cur.execute(query)
                for row in cur.fetchall():
                    all_assets.append(dict(zip(
                        ["asset_id", "last_modified", "last_accessed",
                         "refresh_success_rate", "owner_id", "owner_active"],
                        row
                    )))
        except Exception as e:
            logger.error(f"Skipping {conn_info['host']}: {e}")  # one bad server shouldn't halt the run

    return all_assets

However, reality proved more complicated. Certain content types generated incomplete usage data. Embedded dashboards sometimes produced no meaningful activity signals. Some assets were accessed indirectly through external applications. Others participated in automated processes without receiving direct user traffic.

As a result, we had to move beyond simple metadata checks and evaluate multiple signals simultaneously.

The platform incorporated factors such as:

• Access frequency
• Refresh activity
• Embedded usage patterns
• Ownership status

def staleness_score(asset):
    score = 0
    score += min(asset.days_since_last_access / 90, 1.0) * 0.35
    score += (1 - asset.refresh_success_rate) * 0.25
    score += (0 if asset.embedded_usage_detected else 1) * 0.20
    score += (1 if asset.owner_inactive else 0) * 0.20
    return score  # 0 = active, 1 = strong stale candidate

Rather than relying on a single field, content activity became a composite score built from multiple indicators.

Why a Universal Retention Policy Fails

Another challenge emerged early in development. Initially, we considered applying a universal inactivity threshold to all content. That quickly proved problematic.

Different environments operate under different business rhythms. Development environments tend to have short lifecycles. Financial reporting environments often follow quarterly cycles. Compliance-related assets may only be accessed once per year. Applying the same threshold everywhere creates unnecessary risk.

To address this, we introduced configurable retention policies. Retention could vary based on:

• Environment type
• Project classification
• Known usage patterns
• Business requirements

Environment type	Inactivity threshold before quarantine	Grace period
Development/sandbox	~30 days	3 days
Standard reporting	~90 days	14 days
Financial/quarterly cycles	~120 days	30 days
Compliance/audit	~365 days	45 days

This flexibility allowed lifecycle management to reflect operational reality rather than arbitrary platform rules.

The Importance of the Quarantine Layer

Quarantine became the most important stage of the entire workflow.<