fix: distinguish GPU model suffixes (Ti/Super/XT) and seed RTX 5060 series
RTX 5060 investigation: confirmed via pc-builder.io's own server-side
search (search_query=RTX 5060 against the full 6222-product GPU catalog,
0 results) that it simply isn't in pc-builder.io's catalog yet - not a
scraping bug, the source data doesn't have it. However, while checking
whether the other merchants had it in stock, found a real matching bug:
labelMatchesModelToken bug: extractModelToken("...RTX 5060 Ti...") only
returned "5060", identical to extractModelToken("...RTX 5060...") without
Ti. Every merchant lookup for either card matched on plain "5060" and
could silently return the *other* card's price (confirmed live: LDLC and
Grosbill returned the exact same product/price for both "RTX 5060" and
"RTX 5060 Ti" queries before this fix).
Fixed by:
- extractModelToken now appends a trailing suffix token (Ti, Super, XT,
XTX) when it immediately follows the model number, so "5060 Ti" and
"5060" are extracted as distinct tokens
- New labelMatchesModelToken() replaces the naive
label.includes(modelToken) check everywhere: a bare token ("5060") no
longer matches a label that actually has a suffix right after it
("5060 Ti"), while a suffixed token ("5060 Ti") only matches labels
carrying that exact suffix
- 9 new unit tests covering extraction and matching, including the
regression case (RTX 5060 must not match "RTX 5060 Ti" labels)
Verified fix live: LDLC/Grosbill now return correctly distinct prices for
"RTX 5060" (369.95/364.99) vs "RTX 5060 Ti" (489.95/329.99).
New manual_part_seeder_service.ts: since pc-builder.io's catalog lags
behind real product releases, this seeds a small manually-tracked list of
recent parts it doesn't carry yet (currently RTX 5060 8GB and 5060 Ti
16GB), resolving their price live from the already-supported merchants
rather than hardcoding a number that would go stale. `scrape:parts` now
runs this after the regular catalog scrape. Skips an entry rather than
inserting a guessed price if no merchant has it in stock.
Also fixes a real bug hit while testing the seeder: scraped_parts and
components.purchase_url were varchar(255), which truncates/rejects
Grosbill's long tracking-redirect URLs (500+ characters with encoded
analytics params) - both widened to TEXT via a new migration.
This commit is contained in:
1 parent
21d56a5da4
commit
54c5e37766
5 files changed
+225
-14
No files matched your search
@@ -0,0 +1,27 @@
|
||||
import { BaseSchema } from '@adonisjs/lucid/schema'
|
||||
|
||||
export default class extends BaseSchema {
|
||||
async up() {
|
||||
// Some merchants (Grosbill in particular) return long tracking
|
||||
// redirect URLs with encoded analytics query params well past 255
|
||||
// characters — varchar(255) was truncating/rejecting inserts. Widen
|
||||
// both scraped_parts.source_url (where these URLs are first stored)
|
||||
// and components.purchase_url (where they end up when a scraped part
|
||||
// gets imported into a Component — see components_controller.ts).
|
||||
this.schema.alterTable('scraped_parts', (table) => {
|
||||
table.text('source_url').notNullable().alter()
|
||||
})
|
||||
this.schema.alterTable('components', (table) => {
|
||||
table.text('purchase_url').nullable().alter()
|
||||
})
|
||||
}
|
||||
|
||||
async down() {
|
||||
this.schema.alterTable('scraped_parts', (table) => {
|
||||
table.string('source_url').notNullable().alter()
|
||||
})
|
||||
this.schema.alterTable('components', (table) => {
|
||||
table.string('purchase_url').nullable().alter()
|
||||
})
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user