Package Scores
Socket Scores and Alerts
Socket scores and alerts are broken down into the following categories:
- Supply chain risk
- Quality
- Maintenance
- Vulnerabilities
- License
Socket scores packages across several categories, with alert severity playing a major role. Each type of alert (Critical, High, Medium, or Low) affects the score differently through normalization functions and soft caps. The final score is raised to the power of γ, a scaling factor based on the size and popularity of the project. This exponent compresses low scores and softens the impact of penalties, especially for large or popular packages. Each metric may also optionally apply a limit.
Critical Alerts:
When any critical alerts are present, this metric is limited to the minimum of 0.25 and a weighted average of other scoring signals. The limit of 0.25 acts as a soft cap because the final score is calculated as 100 × (min(limit, weighted-average))^γ. For example, when γ = 0.8, a cap of 0.25 leads to a final score of about 33%. However, if the weighted average is lower than 0.25, the final score can fall even further.
High Alerts:
High alerts reduce the score using an exponential decay, e^−x. The metric is capped at max(0.25, 1 − x/10), bottoming out at 0.25 when there are 8 or more alerts. The actual contribution can go lower due to weighted averaging. If this metric becomes the lowest among all, it can pull down the overall score significantly.
Medium Alerts:
Medium alerts decay more slowly using e^−x/20, with a cap of max(0.5, 1.15 − x/20), which levels off at 0.5 after about 13 alerts. Even though the limit flattens, the normalized value can fall below 0.5 and continue decreasing.
Low Alerts:
Low alerts have the least impact. They decay gently with e^−x/40 and have no cap, so their effect on the score remains small and gradual.
Other Factors:
Package scores are also influenced by project attributes like popularity, size, and maintenance activity. These affect the γ exponent. Larger or more widely used packages tend to have a smaller γ, which reduces the impact of negative metrics and allows those packages to maintain relatively higher scores even when some alerts are present.
Breakdown:
Given a category i, let xi,j be the value of metric j with normalization function Nj and weight wj and limit li,j.
Then the score Si is
Si = 100 · min ( max (0, minj li,j), ∑j wj Nj(xj)/∑j wj )γ
Where γ is a power scaling constant based on the size and popularity of the project:
γ ≈ 1/2 + c0 log(lines of code) + c1 log(popularity)
Currently socket supports the following metrics:
Metric | Category | Weight | Normalization | Limit |
---|---|---|---|---|
Critical Alerts | Any | 1 | e-10x | { 1/4 if x > 0 1 otherwise } |
High Alerts | Any | 2 | e-x | max { 1/4, 1 - x/10 } |
Medium Alerts | Any | 2 | e-x/20 | max { 1/2, 1.15 - x/20 } |
Low Alerts | Any | 3 | e-x/40 | 1 |
License Quality | License | 12 | x/100 | 1 |
Maintainer Count | Maintenance | 5 | -(ex/3 - 1) | { 0 if x = 0 1 otherwise } |
Versions Last Year | Maintenance | 5 | -(e-x/12 - 1) | 1 |
Versions Last Two Months | Maintenance | 3 | -(e-x - 1) | 1 |
Versions Last Month | Maintenance | 2 | -(e-x - 1) | 1 |
Versions Last Week | Maintenance | 1 | -(e-x/0.25 - 1) | 1 |
Open Issues | Maintenance | 1 | e-x/100 | 1 |
Closed Issues | Maintenance | 1 | -(e-x/1000 - 1) | 1 |
Commits Last Week | Maintenance | 1 | -(e-x/4 - 1) | 1 |
Commits Last Month | Maintenance | 1 | -(e-x/6 - 1) | 1 |
Commits Last Two Months | Maintenance | 1 | -(e-x/32 - 1) | 1 |
Commits Last Year | Maintenance | 1 | -(e-x/208 - 1) | 1 |
Commits | Maintenance | 1 | -(e-x/300 - 1) | 1 |
Version Count | Maintenance | 0.5 | -(e-x/10 - 1) | 1 |
Readme Length | Quality | 5 | x/100 | 1 |
Bundle Size | Quality | 2 | e-x/16384 | 1 |
Stargazers | Quality | 1 | -(e-x/100 - 1) | 1 |
Forks | Quality | 1 | -(e-x - 1) | 1 |
Watchers | Quality | 1 | -(e-x - 1) | 1 |
Lines of Code | Quality | 0.5 | x/50 | 1 |
Download Count | Supply Chain Risk | 5 | -(e-x/10000 - 1) | 1 |
Transitive Dependency Count | Supply Chain Risk | 1 | e-x/1000 | 1 |
Total Dependency Count | Supply Chain Risk | 1 | e-x/1000 | 1 |
Dependency Count | Supply Chain Risk | 1 | e-x/50 | 1 |
Dev Dependency Count | Supply Chain Risk | 0.5 | e-x/100 | 1 |
Dependency Vulnerability Count | Vulnerability | 1 | 1 - x | { 0.5 if x > 0 1 otherwise } |
Vulnerability Count | Vulnerability | 1 | 1 - x/3 | 1 |
Please note that these metrics are subject to change and may be revised as we make changes to our system. The contents of this document may not exactly represent the scoring system as deployed in Socket at this point in time as we are continuously making adjustments to these systems.
Updated 25 days ago