Compare commits

..
15 Commits
Author SHA1 Message Date
Kevin L'Hereec 0ce6a7f4a7 fix: clear price to null when a comparison finds no merchant offer
Build & Deploy / build (push) Failing after 3s
Build & Deploy / docker (push) Skipped
Several memory sticks (e.g. Crucial CT8G48C40U5, Patriot Signature Line,
Kingston FURY Beast) kept displaying their pc-builder.io catalog price
("€35.99", "€39.99"...) even after clicking "Compare prices" and getting
"No live price found, using catalog price" - an unconfirmed number was
being shown as if it were still valid, which is misleading given
pc-builder.io's catalog is known to go stale (see the earlier RTX 5070
€128 vs €66.89 case).

New clearPriceIfNoOffers(): when ScrapedPartsController#compare finds
zero offers across all 7 supported merchants, the part's price/priceSource
are reset to null, regardless of whether the price previously came from
the pc-builder.io catalog or an earlier verified merchant match - per
explicit product decision, an empty comparison always wins over keeping a
possibly-stale number around.

- ScrapedPartsController#compare now calls clearPriceIfNoOffers() in the
  empty-offers branch, and returns the resulting price in the JSON
  response so the frontend can mirror it exactly
- NewPartSearch.vue: comparePrices() now sets part.price from the
  response's `price` field (mirroring exactly what the backend just
  persisted) instead of only updating it on a cheaper find; removed the
  now-inaccurate "using catalog price €X" message since that price no
  longer exists once compared with no result
- 3 new tests for clearPriceIfNoOffers: clears an unverified
  pc-builder.io price, also clears a previously-verified merchant price
  when no longer confirmed, and is a no-op when there was no price to
  begin with

Verified end-to-end against the reported Crucial CT8G48C40U5 example:
price went from 35.99 (pc-builder.io) to null after a real comparison
found zero matching offers on any merchant.
2026-07-24 11:20:47 +02:00
Kevin L'Hereec fa74791c92 fix: update displayed catalog price after comparing merchant prices
Build & Deploy / build (push) Failing after 5s
Build & Deploy / docker (push) Skipped
The catalog price shown next to a part (e.g. '€979.95') stayed stuck at
its initial search-result value even after 'Compare prices' found and
persisted a cheaper offer server-side (updatePriceIfCheaper already saves
it to the database, but the locally held search result was never
updated) — so the UI showed a stale, higher price right above a
correctly-labeled 'cheapest' offer, which was confusing (e.g. '€979.95'
next to 'Cdiscount: €741.51 (cheapest)').

comparePrices() now also updates part.price locally when the cheapest
offer found beats it (or no catalog price was known), mirroring what the
backend already persisted, so the display is consistent without needing
to re-run the search.
2026-07-23 17:01:08 +02:00
Kevin L'Hereec 54c5e37766 fix: distinguish GPU model suffixes (Ti/Super/XT) and seed RTX 5060 series
Build & Deploy / build (push) Failing after 3s
Build & Deploy / docker (push) Skipped
RTX 5060 investigation: confirmed via pc-builder.io's own server-side
search (search_query=RTX 5060 against the full 6222-product GPU catalog,
0 results) that it simply isn't in pc-builder.io's catalog yet - not a
scraping bug, the source data doesn't have it. However, while checking
whether the other merchants had it in stock, found a real matching bug:

labelMatchesModelToken bug: extractModelToken("...RTX 5060 Ti...") only
returned "5060", identical to extractModelToken("...RTX 5060...") without
Ti. Every merchant lookup for either card matched on plain "5060" and
could silently return the *other* card's price (confirmed live: LDLC and
Grosbill returned the exact same product/price for both "RTX 5060" and
"RTX 5060 Ti" queries before this fix).

Fixed by:
- extractModelToken now appends a trailing suffix token (Ti, Super, XT,
  XTX) when it immediately follows the model number, so "5060 Ti" and
  "5060" are extracted as distinct tokens
- New labelMatchesModelToken() replaces the naive
  label.includes(modelToken) check everywhere: a bare token ("5060") no
  longer matches a label that actually has a suffix right after it
  ("5060 Ti"), while a suffixed token ("5060 Ti") only matches labels
  carrying that exact suffix
- 9 new unit tests covering extraction and matching, including the
  regression case (RTX 5060 must not match "RTX 5060 Ti" labels)

Verified fix live: LDLC/Grosbill now return correctly distinct prices for
"RTX 5060" (369.95/364.99) vs "RTX 5060 Ti" (489.95/329.99).

New manual_part_seeder_service.ts: since pc-builder.io's catalog lags
behind real product releases, this seeds a small manually-tracked list of
recent parts it doesn't carry yet (currently RTX 5060 8GB and 5060 Ti
16GB), resolving their price live from the already-supported merchants
rather than hardcoding a number that would go stale. `scrape:parts` now
runs this after the regular catalog scrape. Skips an entry rather than
inserting a guessed price if no merchant has it in stock.

Also fixes a real bug hit while testing the seeder: scraped_parts and
components.purchase_url were varchar(255), which truncates/rejects
Grosbill's long tracking-redirect URLs (500+ characters with encoded
analytics params) - both widened to TEXT via a new migration.
2026-07-23 16:51:29 +02:00
Kevin L'Hereec 21d56a5da4 fix: stop protecting stale, unverified pc-builder.io catalog prices
Build & Deploy / build (push) Failing after 3s
Build & Deploy / docker (push) Skipped
Bug found: "Gigabyte GAMING OC GeForce RTX 5070 Ti 16 GB Video Card" kept
showing a stale price of 431.17 even after pc-builder.io itself no longer
had a fixed offer for it (its "Add to Build" HTML only had a generic
Amazon search link, no .price-box element at all, confirmed by fetching
the live page directly).

Root cause: scrapeComponentType's "never lose a better price" rule (added
to make ScrapedPartsController#compare's persisted merchant prices
durable across catalog refreshes) was too broad. It treated ANY existing
price as worth protecting whenever the incoming catalog price was null or
higher - including a price that had only ever come from a *previous*
pc-builder.io scrape itself (priceSource: null), with no independent
verification. Since pc-builder.io's own listings change constantly (a
GPU can lose its priced Amazon link if the referenced ASIN goes out of
stock), that logic effectively froze old pc-builder.io prices forever
even after the site removed them.

Fix: extracted the decision into shouldKeepExistingPrice(), which now
only protects a price when priceSource is a real merchant name (set by
updatePriceIfCheaper after a live comparison confirmed it on the
merchant's own page) - never when priceSource is null (an unverified,
previous pc-builder.io catalog price). A plain re-scrape can always
correct/clear an unverified price, matching whatever pc-builder.io
currently shows.

4 new unit tests for shouldKeepExistingPrice covering: unverified
pc-builder.io price gets overwritten (the regression case), verified
merchant price is protected from worse/missing catalog data, verified
merchant price is still replaced by a genuinely cheaper catalog price,
and no-existing-part returns false.

Also re-ran `node ace scrape:parts` to apply the fix to already-cached
data with stale unverified prices (affected ~1500 rows across the
catalog before the fix, mostly likely unaffected in practice but no
longer silently trusted going forward).
2026-07-23 16:34:48 +02:00
Kevin L'Hereec a1f6ff419a feat: raise search results limit from 30 to 150
Build & Deploy / build (push) Failing after 3s
Build & Deploy / docker (push) Skipped
30 results was too low once pc-builder.io's full catalog is scraped
(e.g. 47 RTX 5070/5070 Ti variants alone), silently truncating results
a user would otherwise see. Also added a result count hint in
NewPartSearch.vue since the list is scrollable (max-h-96) rather than
paginated, so it wasn't obvious more results existed below the fold.
2026-07-23 16:26:30 +02:00
Kevin L'Hereec 6053290501 feat: paginate pc-builder.io scraping and always keep the best known price
Build & Deploy / build (push) Failing after 5s
Build & Deploy / docker (push) Skipped
The RTX 5070 case that prompted this: only 100 GPUs out of pc-builder.io's
actual 6222 were being scraped (its category pages only render the first
page server-side), so cheaper RTX 5070 variants existed on the site but
were invisible to our search. Investigating further also surfaced that
even with full pagination, most recent GPU listings (RTX 5070 included)
have no fixed Amazon price on pc-builder.io at all - just a generic
"search on Amazon" link - so pagination alone doesn't fully solve price
coverage either.

Pagination (part_scraper_service.ts):
- Reverse-engineered pc-builder.io's /products/data endpoint, which powers
  the client-side DataTables widget behind every category page. Requires
  a session cookie + CSRF token scraped from a fresh category page load,
  and an exact field shape (columns[].data must be the literal strings
  "Name"/"Add to Build", not numeric indices - a naive DataTables-shaped
  request 500s otherwise).
- CATEGORY_CONFIG now tracks each category's real pc-builder.io
  category_id and a per-category maxProducts depth, chosen by actual
  catalog size and price-sensitivity: GPU near-full coverage (6000),
  smaller/less price-volatile categories capped lower (300-800) to keep a
  full `scrape:parts` run (all categories) in a reasonable time.
- ensureFreshCategory (triggered inline by a user's search when the cache
  is stale) uses a much shallower QUICK_REFRESH_MAX_PRODUCTS=200 cap
  instead of the category's full depth, since it runs inside the
  request/response cycle of a search - full-depth coverage is only done
  by the standalone `scrape:parts` command.

Always keep the best known price:
- scraped_parts.price is now nullable (migration): parts with no fixed
  pc-builder.io offer are kept and searchable instead of being silently
  skipped, so at least the model is discoverable even before a price is
  resolved.
- New price_source / price_checked_at columns track where the currently
  stored price came from ("pc-builder.io" or a merchant name from
  merchant_price_service.ts).
- ScrapedPartsController#compare now persists the cheapest offer found
  (via the new updatePriceIfCheaper()) when it beats the currently stored
  price - so comparing prices benefits every future search/import for
  that part, not just the user who triggered it. Never regresses the
  price: a comparison finding a higher price than what's stored is
  ignored.
- scrapeComponentType's catalog refresh applies the same "keep whichever
  price is lower" rule per row, so a routine pc-builder.io re-scrape can
  never overwrite a better price a comparison had already found and
  persisted with a worse (or missing) one.
- Search results (GET /scraped-parts) are now ordered priced-cheapest-first,
  then parts with no known price last (`ORDER BY price IS NULL, price ASC`),
  so unpriced parts stay visible without pushing priced results down.
- NewPartSearch.vue displays the catalog price inline (or "No price yet"),
  and the existing "Compare prices" flow now works for those unpriced
  parts the same way it does for priced ones.

7 new/updated functional tests: nullable price is searchable and sorted
last, and 3 tests for updatePriceIfCheaper (cheaper offer replaces price,
pricier offer is ignored, first-ever price is accepted for a previously
unpriced part).
2026-07-23 16:12:26 +02:00
Kevin L'Hereec 9702d24a01 fix: allow gstatic.com in CSP img-src for merchant favicons
Build & Deploy / build (push) Failing after 5s
Build & Deploy / docker (push) Skipped
The favicon service used to show a merchant's icon in the price
comparison UI (www.google.com/s2/favicons) redirects to Google's gstatic
CDN (t0-t3.gstatic.com), which wasn't allowlisted — CSP silently blocked
the redirected image request (reported by some browsers as an opaque
response block rather than a clear CSP violation), so the 'Compare
prices' cheapest-offer favicon never rendered.

Added https://*.gstatic.com to img-src.
2026-07-23 15:38:00 +02:00
Kevin L'Hereec 25d6222df0 fix: add worker-src to CSP to fix broken HMR connection
Build & Deploy / build (push) Failing after 3s
Build & Deploy / docker (push) Skipped
Without an explicit worker-src directive, browsers fall back to
script-src ('self' only) for Web Workers, which blocks the blob: worker
spawned internally by Vite's HMR client / Tuyau's client. This caused the
dev HMR websocket connection to drop intermittently ('server connection
lost. Polling for restart...'), leaving the Vue app in an inconsistent
state after a failed hot reload — components using the Link component
(internally named TuyauLink, from @tuyau/core) would throw 'You must
wrap your app in a TuyauProvider' because the provide/inject context set
up at the app root was lost.

Added worker-src 'self' blob: to config/shield.ts. Verified: dev server
restarts cleanly with the HMR socket connecting without CSP violations,
full test suite (30/30) still passes.
2026-07-23 15:30:01 +02:00
Kevin L'Hereec a7788e593e feat: add Materiel.net, Cdiscount and PCComponentes via headless browser
Build & Deploy / build (push) Failing after 4s
Build & Deploy / docker (push) Skipped
Google/Bing/DuckDuckGo/Qwant were tested as a way to find prices on the
3 remaining requested merchants and all four block scraping immediately
(visible captcha or JS-only anti-bot challenge on the very first
automated request) - not usable as an intermediary. Instead, a headless
browser was used to scrape these 3 merchants directly, since their
blocking (JS-only search, empty body, 403) turned out to be bypassable
with real browser rendering rather than needing to get past a captcha:

- Materiel.net: search is entirely client-side (Stimulus/Turbo), a plain
  GET on /recherche/?query=... always returns the empty "no results"
  template. Fixed by actually filling the search input and clicking
  submit in a real page.
- Cdiscount: returns a fully empty response body to plain HTTP fetches
  (no error, just nothing) but renders fine in a real browser.
- PCComponentes: blocks plain HTTP requests with a 403, but a headless
  browser with a normal user agent goes through.

Implementation:
- New playwright-core dependency, pointed at a system-installed
  Chromium binary via CHROMIUM_EXECUTABLE_PATH (set to /usr/bin/chromium
  here) instead of downloading Playwright's own ~300MB browser
- New headless_browser_service.ts: lazily launches a single shared
  Chromium instance reused across lookups (avoids paying browser launch
  cost per request), with a withPage() helper that always closes its
  browser context afterwards. A new start/browser.ts preload registers
  an app.terminating() hook so the browser process is closed on shutdown
  instead of leaking a child process.
- fetchMaterielNetPrice/fetchCdiscountPrice/fetchPcComponentesPrice
  follow the same model-token matching strategy as the existing
  cheerio-based merchants (LDLC/Grosbill/TopAchat): search on a short
  brand+series query, then pick the result whose label/attribute
  contains the extracted model number, to avoid misreporting an
  unrelated product's price
- PCComponentes exposes clean data-product-price/data-product-category
  attributes directly on result links, so full pre-built "Sobremesa" PCs
  that happen to mention the searched CPU are excluded rather than
  reported as its price
- Fixed two bugs found while testing with a Ryzen 5 9600X (a case with
  multiple search results, unlike the Ryzen 3 3200G single-redirect case
  used earlier):
  - Materiel.net: a search results list page also has .o-product__price
    elements (one per card), so checking for their mere presence isn't
    enough to detect the "redirected to a single product" case; now
    disambiguated by URL pattern (/produit/ vs /recherche/)
  - New parseFlattenedFragmentedPriceText(): Playwright's innerText()
    flattens "<whole>€<sup>cents</sup>" prices to plain strings like
    "284€95" or "2 049€95", which the existing parser misread as
    28495/204995. Cheerio-based scrapers (LDLC/Grosbill) already handled
    this correctly via DOM structure (parseFragmentedPrice); this adds
    the equivalent for already-flattened text.
- findCheapestOffers() now queries all 7 merchants in parallel
- Extended the MerchantOffer type/favicon domain map on the frontend for
  the 3 new merchants

All 30 existing tests still pass unchanged (they don't exercise live
network scraping, only the surrounding auth/validation/caching logic).
2026-07-23 15:25:12 +02:00
Kevin L'Hereec 62ec796e05 fix: repair broken CSP and compare live merchant prices for new parts
Build & Deploy / build (push) Failing after 4s
Build & Deploy / docker (push) Skipped
CSP fix:
- The previous shield.ts config used '@viteDevUrl'/'@viteHmrUrl' CSP
  keywords that don't exist in this version of @adonisjs/shield (only
  '@viteUrl' is registered by @adonisjs/vite, and it resolves to an empty
  string here since Vite assets are proxied through AdonisJS on the same
  origin). Both placeholders were sent to the browser literally, breaking
  Google Fonts, the Vite HMR websocket, and pc-builder.io/favicon images.
- Replaced with concrete directives: 'self' covers same-origin Vite
  assets, ws://localhost:{VITE_HMR_PORT|24678} is added only in dev for
  the HMR socket, and explicit allowlists for Google Fonts
  (fonts.googleapis.com/fonts.gstatic.com), pc-builder.io product images,
  and Google's favicon service (www.google.com) used by the merchant
  comparison UI

Real purchase links + live price comparison:
- part_scraper_service.ts: ScrapedPart.sourceUrl now stores the actual
  Amazon.fr purchase link scraped from pc-builder.io's category pages
  (https://www.amazon.fr/dp/{ASIN}?tag=... or a search fallback), instead
  of the pc-builder.io product page, which isn't a place anyone can buy
  anything from
- New merchant_price_service.ts: on-demand live price lookup across
  Amazon, LDLC, Grosbill and TopAchat. Of 6 requested merchants,
  Materiel.net (search requires JS/Turbo, no results via plain GET) and
  Cdiscount/PCComponentes (empty body / 403) were dropped as not scrapable
  without a headless browser or paid API
- Matching strategy: pc-builder.io's full product names (e.g. "AMD Ryzen 3
  3200G 3.6 GHz Quad-Core Processor") return zero results on every
  merchant's search engine, so we search on a short brand+series query
  and then pick the specific result whose label contains the extracted
  model token (e.g. "3200G"). No token match means no offer returned,
  rather than risk quoting the price of a different, unrelated product
- Fixed price parsing for LDLC/Grosbill, which render prices as nested
  price/sup elements for whole euros vs cents (and LDLC additionally
  wraps that in a duplicate outer .price div) - naive text concatenation
  was misreading "64,95" as "6495"
- New GET /scraped-parts/compare endpoint (authenticated), called on
  demand per search result rather than for every row, since it makes
  several outbound requests and can take a few seconds
- NewPartSearch.vue: "Compare prices" button per result shows the
  cheapest merchant with its favicon and links to the actual offer;
  importing a part now prefers the live cheapest price/link over
  pc-builder.io's potentially stale catalog price (confirmed stale in
  practice: pc-builder.io listed a Ryzen 3 3200G at 128 euros while
  Amazon sells it for 66.89 euros)
- 3 new functional tests for the compare endpoint (auth required, 404 on
  unknown part, 422 on invalid id)
2026-07-23 14:58:33 +02:00
Kevin L'Hereec 4174d92711 feat: track purchase site URL for new components
Build & Deploy / build (push) Failing after 4s
Build & Deploy / docker (push) Skipped
- New nullable purchase_url column on components
- Store/update validators accept an optional purchaseUrl (must be a valid
  absolute URL with a protocol when provided)
- components/create.vue: NewPartSearch import now also fills purchaseUrl
  from the scraped part's source URL (pc-builder.io product page); the
  field remains manually editable for second-hand parts entered by hand
- components/edit.vue: purchaseUrl is editable after creation
- components/show.vue: displays the purchase site as a clickable link
  when present
- 4 new functional tests: create with/without a purchase URL, invalid URL
  rejected (asserts the Inertia inputErrorsBag flash, not a raw 422 -
  content negotiation only returns JSON for XHR/API-style requests), and
  updating an existing component's purchase URL
2026-07-23 14:10:54 +02:00
Kevin L'Hereec 497ae797ca feat: add 24h cache freshness with auto-refresh on search
Build & Deploy / build (push) Failing after 5s
Build & Deploy / docker (push) Skipped
- part_scraper_service.ts: new CACHE_TTL_HOURS (24), isCategoryStale()
  checks the most recent scrapedAt for a given ComponentType's mapped
  category, ensureFreshCategory() re-scrapes it synchronously if missing
  or stale (errors are logged, not thrown, so search still falls back to
  whatever is already cached)
- ScrapedPartsController#index: when a typeId filter is provided, the
  corresponding category cache is refreshed eagerly if stale before
  querying results. Free-text searches without a typeId skip this to
  avoid re-scraping every mapped category on every keystroke -
  'node ace scrape:parts' still refreshes everything at once on demand
- 4 new tests covering staleness detection (missing cache, <24h, >24h)
  and that unmapped types never trigger a scrape/network call
2026-07-23 13:58:37 +02:00
Kevin L'Hereec 5854ff8ed1 feat: search and import new parts scraped from pc-builder.io
Build & Deploy / build (push) Failing after 4s
Build & Deploy / docker (push) Skipped
Idealo and PCPartPicker were evaluated and ruled out:
- idealo.fr search pages return 503 (Cloudflare)
- the internal Idealo GraphQL endpoint used by a public scraper
  (app.idealo.de/app-backend/api) is protected by Akamai Bot Manager and
  silently returns empty data even with mobile app headers
- PCPartPicker's product list/search relies on a session-bound CSRF token
  refreshed per page load, against a scraping-hostile ToS

pc-builder.io category pages are server-rendered, include name/price/image
for ~100 products per category, and have no bot protection, so the feature
is built on top of that source only.

- New scraped_parts cache table (type_id, name, price, image_url, source,
  source_url, source_ref unique per source) populated by a batch job
  rather than scraped live per search, since only ~100 first results per
  category are available without reverse-engineering pc-builder.io's
  DataTables/CSRF pagination flow
- part_scraper_service.ts: maps our ComponentType names to pc-builder.io
  category slugs (SSD -> storage, since pc-builder.io has no separate HDD
  category) and parses each category page with cheerio, skipping rows
  without a resolved price (generic 'search on Amazon' links)
- node ace scrape:parts: refreshes the cache for every mapped category
  (upsert on source+source_ref), runnable manually or later via cron
- GET /scraped-parts (authenticated): search the cache by name (LOWER()
  LIKE, avoids a MySQL utf8_bin/utf8mb4 collation mismatch from Lucid's
  whereLike) and optional typeId filter
- NewPartSearch.vue: debounced search widget wired into
  components/create.vue; clicking 'Import' pre-fills the existing
  component creation form (name, price, type, purchase date = today)
  instead of creating the Component directly, so the user still reviews
  and saves it like any manually entered part
- 3 functional tests covering search by name, filter by typeId, and
  unauthenticated access being redirected to login
- Initial scrape run: 989 parts across 10 categories
2026-07-23 12:10:36 +02:00
Kevin L'Hereec da71d33450 feat: add public read-only sharing for computers
Build & Deploy / build (push) Failing after 5s
Build & Deploy / docker (push) Skipped
Adds the ability for an owner to share a computer build via a public,
unauthenticated link, inspired by a feature request from the original
pcbuilder repo (no such feature actually existed there — implemented
from scratch here).

- New columns on computers: share_token (unique, nullable), share_enabled,
  share_show_sell_price, share_show_cost
- computer_share_service.ts: generates a non-guessable share token
  (crypto.randomBytes, base64url), enables/disables sharing, and supports
  rotating the token to invalidate previously distributed links
- ComputerShareController (authenticated, owner-scoped like the rest of
  the app): PUT /computers/:id/share to update settings, POST
  /computers/:id/share/rotate to rotate the token
- PublicComputerController + GET /share/:token (outside any auth/guest
  group, reachable anonymously): resolves only computers with
  share_enabled = true, 404 otherwise
- ComputerPublicTransformer: dedicated read-only projection that never
  leaks userId, and only includes sellPrice/cost when the owner opted in
  via share_show_sell_price / share_show_cost — everything else (title,
  scores, state, components, pictures) is always visible on a shared link
- ComputerShareModal.vue: owner-facing UI (toggle sharing, choose which
  prices to expose, copy the public link, revoke/rotate it) wired into
  computers/show.vue via a new 'Share' button
- share/show.vue: public page rendered with EmptyLayout (no authenticated
  header/sidebar)
- Boolean columns use a  cast since MySQL returns tinyint(1),
  not native booleans, through Lucid
- 6 new functional tests covering: enabling sharing, price visibility
  toggles, disabling sharing revokes the link, cross-user protection on
  the share settings endpoint (404), token rotation invalidating the old
  link, and anonymous access to a shared computer
2026-07-23 11:43:38 +02:00
Kevin L'Hereec 84ae2b9617 docs: mark remediation plan as executed with implementation summary
Build & Deploy / build (push) Failing after 4s
Build & Deploy / docker (push) Skipped
2026-07-23 11:30:20 +02:00
51 changed files with 3332 additions and 30 deletions

No files matched your search

+5
View File
@@ -6,10 +6,15 @@
/// <reference path="./manifest.d.ts" />
import type { InferData, InferVariants } from '@adonisjs/core/types/transformers'
import type { InferSharedProps } from '@adonisjs/inertia/types'
import type ComputerPublicTransformer from '#transformers/computer_public_transformer'
import type UserTransformer from '#transformers/user_transformer'
import type InertiaMiddleware from '#middleware/inertia_middleware'
export namespace Data {
export type ComputerPublic = InferData<ComputerPublicTransformer>
export namespace ComputerPublic {
export type Variants = InferVariants<ComputerPublicTransformer>
}
export type User = InferData<UserTransformer>
export namespace User {
export type Variants = InferVariants<UserTransformer>
+30
View File
@@ -84,6 +84,24 @@ const routes = {
tokens: [{"old":"/computers/:id","type":0,"val":"computers","end":""},{"old":"/computers/:id","type":1,"val":"id","end":""}],
types: placeholder as Registry['computers.destroy']['types'],
},
'computers.share.update': {
methods: ["PUT"],
pattern: '/computers/:id/share',
tokens: [{"old":"/computers/:id/share","type":0,"val":"computers","end":""},{"old":"/computers/:id/share","type":1,"val":"id","end":""},{"old":"/computers/:id/share","type":0,"val":"share","end":""}],
types: placeholder as Registry['computers.share.update']['types'],
},
'computers.share.rotate': {
methods: ["POST"],
pattern: '/computers/:id/share/rotate',
tokens: [{"old":"/computers/:id/share/rotate","type":0,"val":"computers","end":""},{"old":"/computers/:id/share/rotate","type":1,"val":"id","end":""},{"old":"/computers/:id/share/rotate","type":0,"val":"share","end":""},{"old":"/computers/:id/share/rotate","type":0,"val":"rotate","end":""}],
types: placeholder as Registry['computers.share.rotate']['types'],
},
'share.show': {
methods: ["GET","HEAD"],
pattern: '/share/:token',
tokens: [{"old":"/share/:token","type":0,"val":"share","end":""},{"old":"/share/:token","type":1,"val":"token","end":""}],
types: placeholder as Registry['share.show']['types'],
},
'components.index': {
methods: ["GET","HEAD"],
pattern: '/components',
@@ -126,6 +144,18 @@ const routes = {
tokens: [{"old":"/components/:id","type":0,"val":"components","end":""},{"old":"/components/:id","type":1,"val":"id","end":""}],
types: placeholder as Registry['components.destroy']['types'],
},
'scraped_parts.index': {
methods: ["GET","HEAD"],
pattern: '/scraped-parts',
tokens: [{"old":"/scraped-parts","type":0,"val":"scraped-parts","end":""}],
types: placeholder as Registry['scraped_parts.index']['types'],
},
'scraped_parts.compare': {
methods: ["GET","HEAD"],
pattern: '/scraped-parts/compare',
tokens: [{"old":"/scraped-parts/compare","type":0,"val":"scraped-parts","end":""},{"old":"/scraped-parts/compare","type":0,"val":"compare","end":""}],
types: placeholder as Registry['scraped_parts.compare']['types'],
},
} as const satisfies Record<string, AdonisEndpoint>
export { routes }
+60
View File
@@ -163,6 +163,42 @@ export interface Registry {
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/computers_controller').default['destroy']>>>
}
}
'computers.share.update': {
methods: ["PUT"]
pattern: '/computers/:id/share'
types: {
body: ExtractBody<InferInput<(typeof import('#validators/Computers/share').updateShareSettingsValidator)>>
paramsTuple: [ParamValue]
params: { id: ParamValue }
query: ExtractQuery<InferInput<(typeof import('#validators/Computers/share').updateShareSettingsValidator)>>
response: ExtractResponse<Awaited<ReturnType<import('#controllers/computer_share_controller').default['update']>>>
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/computer_share_controller').default['update']>>> | { status: 422; response: { errors: SimpleError[] } }
}
}
'computers.share.rotate': {
methods: ["POST"]
pattern: '/computers/:id/share/rotate'
types: {
body: {}
paramsTuple: [ParamValue]
params: { id: ParamValue }
query: {}
response: ExtractResponse<Awaited<ReturnType<import('#controllers/computer_share_controller').default['rotate']>>>
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/computer_share_controller').default['rotate']>>>
}
}
'share.show': {
methods: ["GET","HEAD"]
pattern: '/share/:token'
types: {
body: {}
paramsTuple: [ParamValue]
params: { token: ParamValue }
query: {}
response: ExtractResponse<Awaited<ReturnType<import('#controllers/public_computer_controller').default['show']>>>
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/public_computer_controller').default['show']>>>
}
}
'components.index': {
methods: ["GET","HEAD"]
pattern: '/components'
@@ -247,4 +283,28 @@ export interface Registry {
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/components_controller').default['destroy']>>>
}
}
'scraped_parts.index': {
methods: ["GET","HEAD"]
pattern: '/scraped-parts'
types: {
body: {}
paramsTuple: []
params: {}
query: {}
response: ExtractResponse<Awaited<ReturnType<import('#controllers/scraped_parts_controller').default['index']>>>
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/scraped_parts_controller').default['index']>>>
}
}
'scraped_parts.compare': {
methods: ["GET","HEAD"]
pattern: '/scraped-parts/compare'
types: {
body: {}
paramsTuple: []
params: {}
query: {}
response: ExtractResponse<Awaited<ReturnType<import('#controllers/scraped_parts_controller').default['compare']>>>
errorResponse: ExtractErrorResponse<Awaited<ReturnType<import('#controllers/scraped_parts_controller').default['compare']>>>
}
}
}
+11
View File
@@ -20,6 +20,13 @@ export interface ApiDefinition {
edit: typeof routes['computers.edit']
update: typeof routes['computers.update']
destroy: typeof routes['computers.destroy']
share: {
update: typeof routes['computers.share.update']
rotate: typeof routes['computers.share.rotate']
}
}
share: {
show: typeof routes['share.show']
}
components: {
index: typeof routes['components.index']
@@ -30,4 +37,8 @@ export interface ApiDefinition {
update: typeof routes['components.update']
destroy: typeof routes['components.destroy']
}
scrapedParts: {
index: typeof routes['scraped_parts.index']
compare: typeof routes['scraped_parts.compare']
}
}
+3
View File
@@ -5,7 +5,10 @@
export const controllers = {
Components: () => import('#controllers/components_controller'),
ComputerShare: () => import('#controllers/computer_share_controller'),
Computers: () => import('#controllers/computers_controller'),
NewAccount: () => import('#controllers/new_account_controller'),
PublicComputer: () => import('#controllers/public_computer_controller'),
ScrapedParts: () => import('#controllers/scraped_parts_controller'),
Session: () => import('#controllers/session_controller'),
}
+3
View File
@@ -26,15 +26,18 @@ declare module '@adonisjs/inertia/types' {
'layout/emptyLayout': ExtractProps<(typeof import('../../inertia/pages/layout/emptyLayout.vue'))['default']>
'layout/header': ExtractProps<(typeof import('../../inertia/pages/layout/header.vue'))['default']>
'layout/sidebar': ExtractProps<(typeof import('../../inertia/pages/layout/sidebar.vue'))['default']>
'share/show': ExtractProps<(typeof import('../../inertia/pages/share/show.vue'))['default']>
'widgets/AuthErrorAlert': ExtractProps<(typeof import('../../inertia/pages/widgets/AuthErrorAlert.vue'))['default']>
'widgets/ComponentEditor': ExtractProps<(typeof import('../../inertia/pages/widgets/ComponentEditor.vue'))['default']>
'widgets/ComponentsViewer': ExtractProps<(typeof import('../../inertia/pages/widgets/ComponentsViewer.vue'))['default']>
'widgets/ComputerCard': ExtractProps<(typeof import('../../inertia/pages/widgets/ComputerCard.vue'))['default']>
'widgets/ComputerComponentsTable': ExtractProps<(typeof import('../../inertia/pages/widgets/ComputerComponentsTable.vue'))['default']>
'widgets/ComputerShareModal': ExtractProps<(typeof import('../../inertia/pages/widgets/ComputerShareModal.vue'))['default']>
'widgets/ComputersViewer': ExtractProps<(typeof import('../../inertia/pages/widgets/ComputersViewer.vue'))['default']>
'widgets/CustomSelect': ExtractProps<(typeof import('../../inertia/pages/widgets/CustomSelect.vue'))['default']>
'widgets/DeleteConfirmationModal': ExtractProps<(typeof import('../../inertia/pages/widgets/DeleteConfirmationModal.vue'))['default']>
'widgets/FilterDropdown': ExtractProps<(typeof import('../../inertia/pages/widgets/FilterDropdown.vue'))['default']>
'widgets/NewPartSearch': ExtractProps<(typeof import('../../inertia/pages/widgets/NewPartSearch.vue'))['default']>
'widgets/SimpleGallery': ExtractProps<(typeof import('../../inertia/pages/widgets/SimpleGallery.vue'))['default']>
'widgets/TableComponent': ExtractProps<(typeof import('../../inertia/pages/widgets/TableComponent.vue'))['default']>
'widgets/ValidationError': ExtractProps<(typeof import('../../inertia/pages/widgets/ValidationError.vue'))['default']>
+13
View File
@@ -17,6 +17,9 @@ export type ScannedRoutes = {
'computers.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.destroy': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.share.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.share.rotate': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'share.show': { paramsTuple: [ParamValue]; params: {'token': ParamValue} }
'components.index': { paramsTuple?: []; params?: {} }
'components.create': { paramsTuple?: []; params?: {} }
'components.store': { paramsTuple?: []; params?: {} }
@@ -24,6 +27,8 @@ export type ScannedRoutes = {
'components.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.destroy': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'scraped_parts.index': { paramsTuple?: []; params?: {} }
'scraped_parts.compare': { paramsTuple?: []; params?: {} }
}
GET: {
'home': { paramsTuple?: []; params?: {} }
@@ -33,10 +38,13 @@ export type ScannedRoutes = {
'computers.create': { paramsTuple?: []; params?: {} }
'computers.show': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'share.show': { paramsTuple: [ParamValue]; params: {'token': ParamValue} }
'components.index': { paramsTuple?: []; params?: {} }
'components.create': { paramsTuple?: []; params?: {} }
'components.show': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'scraped_parts.index': { paramsTuple?: []; params?: {} }
'scraped_parts.compare': { paramsTuple?: []; params?: {} }
}
HEAD: {
'home': { paramsTuple?: []; params?: {} }
@@ -46,20 +54,25 @@ export type ScannedRoutes = {
'computers.create': { paramsTuple?: []; params?: {} }
'computers.show': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'share.show': { paramsTuple: [ParamValue]; params: {'token': ParamValue} }
'components.index': { paramsTuple?: []; params?: {} }
'components.create': { paramsTuple?: []; params?: {} }
'components.show': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.edit': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'scraped_parts.index': { paramsTuple?: []; params?: {} }
'scraped_parts.compare': { paramsTuple?: []; params?: {} }
}
POST: {
'new_account.store': { paramsTuple?: []; params?: {} }
'session.store': { paramsTuple?: []; params?: {} }
'session.destroy': { paramsTuple?: []; params?: {} }
'computers.store': { paramsTuple?: []; params?: {} }
'computers.share.rotate': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.store': { paramsTuple?: []; params?: {} }
}
PUT: {
'computers.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'computers.share.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
'components.update': { paramsTuple: [ParamValue]; params: {'id': ParamValue} }
}
DELETE: {
+7 -1
View File
@@ -18,4 +18,10 @@ DB_HOST=127.0.0.1
DB_PORT=3306
DB_USER=admin
DB_PASSWORD=password
DB_DATABASE=pc_builderv2
DB_DATABASE=pc_builderv2
# Merchant price comparison (headless browser scraping via playwright-core)
# Path to a system Chromium/Chrome binary. Leave unset to let Playwright
# try its own bundled browser (must be installed separately with
# `npx playwright install chromium` if so).
CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium
+43
View File
@@ -1,5 +1,8 @@
# Plan de remédiation — PC Builder
> **Statut : exécuté le 2026-07-23.** Voir le résumé d'implémentation à la
> fin de ce document pour le détail des commits et des décisions prises.
Basé sur la revue de code du 2026-07-23. Objectif : corriger les failles de
sécurité critiques avant toute mise en production, puis assainir la base du
code (majeur), puis traiter les points mineurs/qualité.
@@ -318,3 +321,43 @@ fonctionnels Japa :
- **2.5** : garder, implémenter ou supprimer le champ `origin`.
- **2.8** : supprimer les composants orphelins (Option A) ou finaliser la
fonctionnalité galerie/éditeur de composants uniques (Option B).
---
## Résumé d'implémentation (2026-07-23)
Toutes les phases du plan ont été exécutées. Commits (après réécriture
d'historique, voir 1.3 ci-dessous) :
- `3cc189c` — fix(security): close IDOR on computers/components update & destroy
(Phase 1: 1.1, 1.2, 1.5, 2.1, 2.2, retrait du suivi de `.env.production`)
- `3579c9d` — refactor: uniform validation, fix create form UX, dedupe filter
dropdowns (Phase 2: 2.3, 2.4, 2.5, 2.7, déplacement `MIGRATION_*.md`)
- `94f89be` — chore: harden config, strengthen frontend typing, fix UTF-8
mojibake (Phase 3: 3.1, 3.2, 3.3, 3.5)
- `562cea3` — test: add functional regression tests for IDOR fixes (Phase 4)
- `b888718` — docs(ci): flag NAS deploy secrets risk and .env.production
breakage (documentation de 1.4)
### Décisions prises avec l'utilisateur
- **1.3 (purge historique)** : `git-filter-repo` a été utilisé pour retirer
`.env.production` de tous les commits, une sauvegarde bundle complète a été
créée avant l'opération (`/tmp/user/1000/kilo/pc_builder_backup_*.bundle`),
puis l'historique réécrit a été poussé en force vers `origin/main`. Tout
autre clone existant du dépôt doit être resynchronisé
(`git fetch && git reset --hard origin/main`). **Les secrets réels
(APP_KEY, DB_PASSWORD du NAS) doivent encore être régénérés manuellement**,
car ils ont circulé en clair dans l'historique avant la purge.
- **1.4 (CI/NAS)** : documenté uniquement dans `.gitea/workflows/build.yml`
(pas d'accès au NAS pour reconfigurer SSH/sudo). Le retrait de
`.env.production` du suivi git casse le step `cat .env.production` du
pipeline de déploiement — c'est documenté en commentaire, à corriger en
migrant les secrets vers Gitea Actions.
- **2.5 (champ `origin`)** : supprimé (jamais fourni par le backend).
- **2.8 (widgets orphelins)** : conservés tels quels, non supprimés.
### Non traité
- Phase 2.6 a été traitée dans le commit de sécurité (détachement des
composants + nettoyage des fichiers uploads à la suppression d'un
ordinateur), avancée par rapport au planning initial.
- 2.8 tranché "garder tel quel" — aucune action requise.
+1
View File
@@ -73,6 +73,7 @@ export default defineConfig({
() => import('#start/routes'),
() => import('#start/kernel'),
() => import('#start/validator'),
() => import('#start/browser'),
],
/*
@@ -0,0 +1,46 @@
import type { HttpContext } from '@adonisjs/core/http'
import Computer from '#models/computer'
import { updateShareSettingsValidator } from '#validators/Computers/share'
import { disableSharing, enableSharing, rotateShareToken } from '#services/computer_share_service'
export default class ComputerShareController {
/**
* Update the share settings (enable/disable + which prices are visible)
* for a computer owned by the current user.
*/
async update({ auth, params, request, response }: HttpContext) {
const user = await auth.authenticate()
const computer = await Computer.query()
.where('user_id', user.id)
.andWhere('id', params.id)
.firstOrFail()
const { enabled, showSellPrice, showCost } = await request.validateUsing(
updateShareSettingsValidator
)
if (enabled) {
await enableSharing(computer, { showSellPrice, showCost })
} else {
await disableSharing(computer)
}
return response.redirect(`/computers/${computer.id}`)
}
/**
* Rotate the share token for a computer, invalidating any previously
* distributed public link.
*/
async rotate({ auth, params, response }: HttpContext) {
const user = await auth.authenticate()
const computer = await Computer.query()
.where('user_id', user.id)
.andWhere('id', params.id)
.firstOrFail()
await rotateShareToken(computer)
return response.redirect(`/computers/${computer.id}`)
}
}
+3 -1
View File
@@ -80,7 +80,9 @@ export default class ComputersController {
.where('user_id', user.id)
.andWhere('id', computerId)
.firstOrFail()
return inertia.render('computers/show', { computer: asPageProp(computer.serialize()) })
return inertia.render('computers/show', {
computer: asPageProp({ ...computer.serialize(), shareToken: computer.shareToken }),
})
}
async edit({ params, inertia, auth }: HttpContext) {
@@ -0,0 +1,32 @@
import type { HttpContext } from '@adonisjs/core/http'
import Computer from '#models/computer'
import ComputerPublicTransformer from '#transformers/computer_public_transformer'
export default class PublicComputerController {
/**
* Publicly display a computer via its share token. No authentication
* required. Only resolves computers with sharing explicitly enabled.
*/
async show({ params, inertia, response }: HttpContext) {
const computer = await Computer.query()
.where('share_token', params.token)
.andWhere('share_enabled', true)
.preload('state')
.preload('components', (componentsQuery) => {
componentsQuery.preload('type')
})
.preload('pictures', (picturesQuery) => {
picturesQuery.orderBy('order')
})
.first()
if (!computer) {
response.status(404)
return inertia.render('errors/not_found', {})
}
return inertia.render('share/show', {
computer: ComputerPublicTransformer.transform(computer),
})
}
}
+109
View File
@@ -0,0 +1,109 @@
import type { HttpContext } from '@adonisjs/core/http'
import vine from '@vinejs/vine'
import ScrapedPart from '#models/scraped_part'
import ComponentType from '#models/component_type'
import {
ensureFreshCategory,
updatePriceIfCheaper,
clearPriceIfNoOffers,
} from '#services/part_scraper_service'
import { findCheapestOffers } from '#services/merchant_price_service'
const SEARCH_RESULTS_LIMIT = 150
const searchValidator = vine.compile(
vine.object({
q: vine.string().trim().minLength(1).optional(),
typeId: vine.number().min(1).optional(),
})
)
const compareValidator = vine.compile(
vine.object({
id: vine.number().min(1),
})
)
export default class ScrapedPartsController {
/**
* Search the new-parts cache (scraped from pc-builder.io) by name and/or
* component type. Read-only, authenticated (the search UI lives inside
* the authenticated "create component" flow).
*
* When a typeId is given, the corresponding category cache is refreshed
* synchronously first if it's missing or older than 24h (see
* part_scraper_service.CACHE_TTL_HOURS), so results stay reasonably
* fresh without needing an external cron job. Free-text searches across
* all types skip this eager refresh to avoid re-scraping every mapped
* category on every keystroke; run `node ace scrape:parts` to refresh
* everything at once.
*
* Results are ordered with a resolved price first (cheapest first), then
* parts with no known price yet (see ScrapedPart.price doc) last — so a
* part is never hidden just because pc-builder.io hasn't found a fixed
* Amazon offer for it, but priced parts still surface first.
*/
async index({ request, auth, response }: HttpContext) {
await auth.authenticate()
const { q, typeId } = await request.validateUsing(searchValidator)
if (typeId) {
const componentType = await ComponentType.find(typeId)
if (componentType) {
await ensureFreshCategory(componentType)
}
}
const query = ScrapedPart.query()
.preload('type')
.orderByRaw('price IS NULL, price ASC, name ASC')
if (q) {
query.whereRaw('LOWER(name) LIKE ?', [`%${q.toLowerCase()}%`])
}
if (typeId) {
query.where('typeId', typeId)
}
const parts = await query.limit(SEARCH_RESULTS_LIMIT)
return response.json(parts.map((part) => part.serialize()))
}
/**
* Compares live prices for a given scraped part across Amazon, LDLC,
* Grosbill, TopAchat, Materiel.net, Cdiscount and PCComponentes, and
* returns them sorted cheapest first. Triggered on demand (one part at a
* time) rather than for every search result, since it makes several
* outbound requests and can take a few seconds.
*
* If the cheapest offer found beats the part's currently stored price
* (or no price was known yet), it's persisted as the new default price
* (see updatePriceIfCheaper) — so this comparison benefits every future
* search/import for this part, not just the current user.
*
* If NO merchant confirms a price at all, the part's price is cleared
* to null (see clearPriceIfNoOffers) rather than silently keeping a
* pc-builder.io catalog price the user just explicitly asked to verify
* and that no longer checks out anywhere.
*/
async compare({ request, auth, response }: HttpContext) {
await auth.authenticate()
const { id } = await request.validateUsing(compareValidator)
const part = await ScrapedPart.find(id)
if (!part) {
return response.notFound({ offers: [] })
}
const amazonUrl = part.sourceUrl.includes('amazon.') ? part.sourceUrl : null
const offers = await findCheapestOffers(part.name, amazonUrl)
if (offers.length > 0) {
await updatePriceIfCheaper(part, offers[0])
} else {
await clearPriceIfNoOffers(part)
}
return response.json({ offers, price: part.price })
}
}
+3
View File
@@ -24,6 +24,9 @@ export default class Component extends BaseModel {
@column()
declare computerId: number | null
@column()
declare purchaseUrl: string | null
@column({ serializeAs: null })
declare userId: number
+12
View File
@@ -44,6 +44,18 @@ export default class Computer extends BaseModel {
@column({ serializeAs: null })
declare userId: number
@column({ serializeAs: null })
declare shareToken: string | null
@column({ consume: (value: unknown) => Boolean(value) })
declare shareEnabled: boolean
@column({ consume: (value: unknown) => Boolean(value) })
declare shareShowSellPrice: boolean
@column({ consume: (value: unknown) => Boolean(value) })
declare shareShowCost: boolean
@column.dateTime({ autoCreate: true })
declare createdAt: DateTime
+11
View File
@@ -0,0 +1,11 @@
import { belongsTo } from '@adonisjs/lucid/orm'
import type { BelongsTo } from '@adonisjs/lucid/types/relations'
import { ScrapedPartSchema } from '#database/schema'
import ComponentType from '#models/component_type'
export default class ScrapedPart extends ScrapedPartSchema {
@belongsTo(() => ComponentType, {
foreignKey: 'typeId',
})
declare type: BelongsTo<typeof ComponentType>
}
+50
View File
@@ -0,0 +1,50 @@
import { randomBytes } from 'node:crypto'
import Computer from '#models/computer'
/**
* Generates a URL-safe, non-guessable share token.
*/
function generateShareToken(): string {
return randomBytes(24).toString('base64url')
}
/**
* Enables sharing for a computer, generating a share token if it doesn't
* already have one. Idempotent: calling it again just updates the display
* options without rotating the existing token.
*/
async function enableSharing(
computer: Computer,
options: { showSellPrice: boolean; showCost: boolean }
) {
if (!computer.shareToken) {
computer.shareToken = generateShareToken()
}
computer.shareEnabled = true
computer.shareShowSellPrice = options.showSellPrice
computer.shareShowCost = options.showCost
await computer.save()
return computer
}
/**
* Disables sharing for a computer. The share token is kept so re-enabling
* later doesn't require distributing a new link, but the public route only
* resolves computers where shareEnabled is true.
*/
async function disableSharing(computer: Computer) {
computer.shareEnabled = false
await computer.save()
return computer
}
/**
* Rotates the share token, invalidating any previously distributed link.
*/
async function rotateShareToken(computer: Computer) {
computer.shareToken = generateShareToken()
await computer.save()
return computer
}
export { enableSharing, disableSharing, rotateShareToken }
+69
View File
@@ -0,0 +1,69 @@
import { chromium, type Browser } from 'playwright-core'
import env from '#start/env'
import logger from '@adonisjs/core/services/logger'
const USER_AGENT =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
let browserPromise: Promise<Browser> | null = null
/**
* Lazily launches a single shared headless Chromium instance, reused
* across requests instead of paying the ~1-2s browser startup cost on
* every merchant lookup. Points at a system-installed Chromium/Chrome
* binary via CHROMIUM_EXECUTABLE_PATH (see .env.example) rather than
* bundling Playwright's own ~300MB download.
*/
function getBrowser(): Promise<Browser> {
if (!browserPromise) {
browserPromise = chromium
.launch({
executablePath: env.get('CHROMIUM_EXECUTABLE_PATH') || undefined,
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox'],
})
.catch((error) => {
browserPromise = null
throw error
})
}
return browserPromise
}
/**
* Runs `fn` with a fresh browser page/context, always closing the context
* afterwards (success or failure) so pages don't accumulate across many
* lookups. The underlying browser process itself stays warm between calls.
* Returns null (rather than throwing) if the browser fails to launch or the
* callback throws, since merchant lookups are best-effort.
*/
async function withPage<T>(
fn: (page: import('playwright-core').Page) => Promise<T>
): Promise<T | null> {
try {
const browser = await getBrowser()
const context = await browser.newContext({ userAgent: USER_AGENT, locale: 'fr-FR' })
try {
const page = await context.newPage()
return await fn(page)
} finally {
await context.close()
}
} catch (error) {
logger.warn({ err: error }, 'Headless browser lookup failed')
return null
}
}
/**
* Closes the shared browser instance, if running. Intended for graceful
* shutdown hooks / tests, not needed during normal request handling.
*/
async function closeBrowser(): Promise<void> {
if (!browserPromise) return
const browser = await browserPromise.catch(() => null)
browserPromise = null
await browser?.close()
}
export { withPage, closeBrowser }
@@ -0,0 +1,82 @@
import { DateTime } from 'luxon'
import logger from '@adonisjs/core/services/logger'
import ComponentType from '#models/component_type'
import ScrapedPart from '#models/scraped_part'
import { findCheapestOffers } from '#services/merchant_price_service'
const SOURCE = 'manual'
/**
* Very recent parts pc-builder.io doesn't carry yet (its catalog appears
* to lag several months behind actual product releases — confirmed via
* its own server-side search returning 0 results for "RTX 5060"). These
* are seeded manually so they're at least searchable/importable, with
* their price resolved live from the merchants merchant_price_service.ts
* already supports (LDLC/Grosbill both carry the RTX 5060 series, for
* example) rather than a hardcoded number that would go stale immediately.
*
* sourceRef must be stable and unique per entry (used as the upsert key,
* like pc-builder.io's own product ID), and is namespaced under
* "manual-" to guarantee it can never collide with a numeric
* pc-builder.io data-id.
*/
const MANUAL_PARTS: { typeName: string; name: string; sourceRef: string }[] = [
{
typeName: 'Graphic Card',
name: 'NVIDIA GeForce RTX 5060 8 GB Video Card',
sourceRef: 'manual-rtx-5060-8gb',
},
{
typeName: 'Graphic Card',
name: 'NVIDIA GeForce RTX 5060 Ti 16 GB Video Card',
sourceRef: 'manual-rtx-5060-ti-16gb',
},
]
/**
* Upserts every entry in MANUAL_PARTS, resolving each one's price live
* from the supported merchants (Amazon/LDLC/Grosbill/...). An entry with
* no ComponentType match in the current database, or no merchant offer
* found at all, is skipped rather than inserted with a null/guessed
* price — this only fills in real, verified prices.
*/
async function seedManualParts(): Promise<{ inserted: number; skipped: number }> {
let inserted = 0
let skipped = 0
for (const manualPart of MANUAL_PARTS) {
const componentType = await ComponentType.findBy('name', manualPart.typeName)
if (!componentType) {
logger.warn({ part: manualPart.name }, 'No matching ComponentType, skipping manual part')
skipped++
continue
}
const offers = await findCheapestOffers(manualPart.name, null)
if (offers.length === 0) {
logger.warn({ part: manualPart.name }, 'No merchant offer found, skipping manual part')
skipped++
continue
}
const cheapest = offers[0]
await ScrapedPart.updateOrCreate(
{ source: SOURCE, sourceRef: manualPart.sourceRef },
{
typeId: componentType.id,
name: manualPart.name,
imageUrl: null,
price: cheapest.price,
sourceUrl: cheapest.url,
priceSource: cheapest.merchant,
priceCheckedAt: DateTime.now(),
scrapedAt: DateTime.now(),
}
)
inserted++
}
return { inserted, skipped }
}
export { seedManualParts, MANUAL_PARTS }
+525
View File
@@ -0,0 +1,525 @@
import * as cheerio from 'cheerio'
import logger from '@adonisjs/core/services/logger'
import { withPage } from '#services/headless_browser_service'
const USER_AGENT =
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
export interface MerchantOffer {
merchant:
| 'Amazon'
| 'LDLC'
| 'Grosbill'
| 'TopAchat'
| 'Materiel.net'
| 'Cdiscount'
| 'PCComponentes'
price: number
url: string
}
async function fetchHtml(url: string): Promise<string | null> {
try {
const response = await fetch(url, {
headers: {
'User-Agent': USER_AGENT,
'Accept-Language': 'fr-FR,fr;q=0.9',
},
})
if (!response.ok) return null
return await response.text()
} catch (error) {
logger.warn({ url, err: error }, 'Failed to fetch merchant page')
return null
}
}
/**
* Parses a price from plain text like "64,95 €" or "413.59 €*".
*/
function parsePriceText(text: string): number | null {
const normalized = text
.replace(/\u00a0/g, ' ')
.replace(/[^\d,.\s]/g, '')
.trim()
.replace(/\s/g, '')
.replace(',', '.')
const value = Number.parseFloat(normalized)
return Number.isNaN(value) ? null : value
}
/**
* Some merchants (LDLC, Grosbill, Materiel.net) render prices split across
* sibling/child elements instead of plain text, e.g.
* `<div class="price">64€<sup>95</sup></div>` renders as "64€95" once
* flattened to text, which parsePriceText would misread as 6495 (or, worse,
* "2 049€95" as 204995). This detects that "<whole>€<cents>" shape in
* flattened text and recombines it as <whole>.<cents> instead.
*/
function parseFlattenedFragmentedPriceText(text: string): number | null {
const match = text
.replace(/\u00a0/g, ' ')
.trim()
.match(/^([\d\s]+)\s*€\s*(\d{1,2})$/)
if (!match) return null
const whole = match[1].replace(/\s/g, '')
const cents = match[2].padEnd(2, '0').slice(0, 2)
const value = Number.parseFloat(`${whole}.${cents}`)
return Number.isNaN(value) ? null : value
}
/**
* Same fragmented-price shape as parseFlattenedFragmentedPriceText, but
* operating on a Cheerio element with a real `<sup>` child instead of
* already-flattened text — used when scraping static HTML with cheerio
* (LDLC, Grosbill) rather than reading `.innerText()` from a live browser
* page (Materiel.net).
*
* LDLC additionally nests a duplicate `.price` wrapper
* (`<div class="price"><div class="price">64€<sup>95</sup></div></div>`),
* so this always resolves to the innermost element actually containing a
* `<sup>`, rather than assuming $el itself is the leaf node.
*/
function parseFragmentedPrice($el: cheerio.Cheerio<any>): number | null {
const sup = $el.find('sup').first()
if (sup.length > 0) {
// The element that directly holds both the whole-part text node and
// the <sup> cents — not necessarily $el itself if it's a wrapper.
const leaf = sup.parent()
const cents = sup.text().replace(/\D/g, '')
const wholePart = leaf.clone().children().remove().end().text().replace(/\D/g, '')
if (wholePart) {
const value = Number.parseFloat(`${wholePart}.${cents.padEnd(2, '0').slice(0, 2)}`)
return Number.isNaN(value) ? null : value
}
}
return parsePriceText($el.text())
}
/**
* Builds a short, merchant-search-friendly query from a full pc-builder.io
* product name. These sites' search engines are strict/exact-match and
* choke on the long, spec-heavy names pc-builder.io uses (e.g. "AMD Ryzen 3
* 3200G 3.6 GHz Quad-Core Processor" returns zero results everywhere, but
* "AMD Ryzen 3" does). We search on the first few tokens (brand + series),
* then use extractModelToken() to pick out the right result from the list.
*/
function shortSearchQuery(name: string): string {
const tokens = name
.replace(/\([^)]*\)/g, ' ')
.split(/\s+/)
.filter(Boolean)
return tokens.slice(0, 3).join(' ')
}
/**
* Suffix tokens that modify a base GPU/CPU model number into a distinct
* SKU with a different price (e.g. "RTX 5060" vs "RTX 5060 Ti", "RTX 4070"
* vs "RTX 4070 Super"). When immediately following the model token, these
* must be treated as part of it — otherwise "5060 Ti" only matches on
* "5060" and can't be told apart from plain "5060".
*/
const MODEL_SUFFIXES = ['Ti', 'Super', 'XT', 'XTX']
/**
* Extracts the most specific "model number" token from a product name, to
* disambiguate between multiple search results (e.g. "3200G" out of "AMD
* Ryzen 3 3200G ...", "970" out of "Samsung 970 Evo Plus ...", or
* "5060 Ti" out of "NVIDIA GeForce RTX 5060 Ti 16 GB ..." — as opposed to
* plain "5060", which is a different, cheaper card). Prefers alphanumeric
* tokens that mix letters and digits (typical of CPU/GPU model numbers),
* falling back to a bare 3-4 digit number.
*/
function extractModelToken(name: string): string | null {
const tokens = name
.replace(/\([^)]*\)/g, ' ')
.split(/\s+/)
.filter(Boolean)
const matchWithSuffix = (index: number, base: string): string => {
const next = tokens[index + 1]
return next && MODEL_SUFFIXES.includes(next) ? `${base} ${next}` : base
}
const mixedIndex = tokens.findIndex(
(t) => /[a-zA-Z]/.test(t) && /\d/.test(t) && !/^\d+(\.\d+)?ghz$/i.test(t)
)
if (mixedIndex !== -1) return matchWithSuffix(mixedIndex, tokens[mixedIndex])
const bareIndex = tokens.findIndex((t) => /^\d{3,4}$/.test(t))
if (bareIndex !== -1) return matchWithSuffix(bareIndex, tokens[bareIndex])
return null
}
/**
* True if `label` matches `modelToken` as the specific SKU it represents,
* not just as a loose substring. Plain string.includes() would let "5060"
* match "RTX 5060 Ti" (wrong, more expensive/different card) and vice
* versa "5060 Ti" would still substring-match "5060" only by accident of
* word order — this instead requires the model token to appear as its own
* word boundary, and additionally rejects a bare model number (no suffix)
* when the label actually has one of MODEL_SUFFIXES right after it.
*/
function labelMatchesModelToken(label: string, modelToken: string): boolean {
const normalizedLabel = label.toLowerCase()
const normalizedToken = modelToken.toLowerCase()
if (!normalizedLabel.includes(normalizedToken)) return false
const hasSuffix = normalizedToken.includes(' ')
if (hasSuffix) return true
// Bare token (e.g. "5060"): reject if the label has it immediately
// followed by a known suffix (e.g. "5060 Ti"), since that's a distinct,
// different-priced SKU we should not conflate with the plain model.
const suffixPattern = MODEL_SUFFIXES.join('|')
const followedBySuffix = new RegExp(`\\b${normalizedToken}\\s+(${suffixPattern})\\b`, 'i')
return !followedBySuffix.test(label)
}
/**
* Reads the price directly from a known Amazon.fr product page
* (https://www.amazon.fr/dp/{ASIN}?...). Only works for direct product
* links; Amazon search-result links (https://www.amazon.fr/s?k=...) can't
* be resolved to a single price this way.
*/
async function fetchAmazonPrice(amazonUrl: string): Promise<MerchantOffer | null> {
if (!amazonUrl.includes('/dp/')) return null
const html = await fetchHtml(amazonUrl)
if (!html) return null
const $ = cheerio.load(html)
const priceText = $('.a-price .a-offscreen').first().text().trim()
const price = parsePriceText(priceText)
if (price === null) return null
return { merchant: 'Amazon', price, url: amazonUrl }
}
/**
* Searches LDLC by product name and reads the price of the result whose
* label contains the extracted model token (e.g. "3200G"). Only falls back
* to the first result when there is no model token to disambiguate with
* (rare); otherwise a search that returns unrelated products (LDLC simply
* doesn't carry this exact model) correctly yields no offer instead of a
* misleading price for the wrong product.
*/
async function fetchLdlcPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
const url = `https://www.ldlc.com/recherche/${encodeURIComponent(query)}/`
const html = await fetchHtml(url)
if (!html) return null
const $ = cheerio.load(html)
const items = $('.listing-product li.pdt-item')
if (!items.length) return null
let best: cheerio.Cheerio<any> | null = modelToken ? null : items.first()
if (modelToken) {
items.each((_, el) => {
const label = $(el).find('.pdt-desc a').first().text()
if (labelMatchesModelToken(label, modelToken)) {
best = $(el)
return false
}
})
}
if (!best) return null
const link = (best as cheerio.Cheerio<any>).find('.pdt-desc a').first().attr('href')
const priceEl = (best as cheerio.Cheerio<any>).find('.price').first()
const price = parseFragmentedPrice(priceEl)
if (price === null || !link) return null
return { merchant: 'LDLC', price, url: new URL(link, 'https://www.ldlc.com').toString() }
}
/**
* Searches Grosbill by product name and reads the price of the result
* whose label contains the extracted model token. Same rationale as
* fetchLdlcPrice: without a model token match, we return null rather than
* risk quoting the price of an unrelated product.
*/
async function fetchGrosbillPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
const url = `https://www.grosbill.com/produit.aspx?q=${encodeURIComponent(query)}/`
const html = await fetchHtml(url)
if (!html) return null
const $ = cheerio.load(html)
const items = $('[class*="la_liste_des_produits"]')
if (!items.length) return null
let best: cheerio.Cheerio<any> | null = modelToken ? null : items.first()
if (modelToken) {
items.each((_, el) => {
const label = $(el)
.find('.grb__liste-produit__liste__produit__information__libelle')
.text()
.replace(/\s+/g, ' ')
if (labelMatchesModelToken(label, modelToken)) {
best = $(el)
return false
}
})
}
if (!best) return null
const priceEl = (best as cheerio.Cheerio<any>).find('.price-main').first()
const price = parseFragmentedPrice(priceEl)
if (price === null) return null
const link = (best as cheerio.Cheerio<any>).find('a[href]').first().attr('href')
return { merchant: 'Grosbill', price, url: link ? new URL(link, url).toString() : url }
}
/**
* Searches TopAchat by product name. When the search resolves to a single
* matching product, TopAchat redirects straight to the product page
* (label-product / offer-price__price). List-page matching by model token
* is attempted first for queries with multiple hits.
*/
async function fetchTopAchatPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
const url = `https://www.topachat.com/search/${encodeURIComponent(query)}`
const html = await fetchHtml(url)
if (!html) return null
const $ = cheerio.load(html)
// Case 1: redirected straight to a single product page.
const singlePrice = $('.offer-price__price').first()
if (singlePrice.length) {
const label = $('.label-product').first().text()
if (!modelToken || labelMatchesModelToken(label, modelToken)) {
const priceText = singlePrice.clone().children().remove().end().text().trim()
const price = parsePriceText(priceText)
if (price !== null) {
const canonical = $('link[rel="canonical"]').attr('href')
return { merchant: 'TopAchat', price, url: canonical || url }
}
}
}
// Case 2: a list of results (product__price / product__label cards).
const cards = $('article.product, [class*="product__"]').filter(
(_, el) => $(el).find('.product__price').length > 0
)
let best: cheerio.Cheerio<any> | null = null
cards.each((_, el) => {
const label = $(el).find('.product__label').text()
if (!modelToken || labelMatchesModelToken(label, modelToken)) {
best = $(el)
return false
}
})
if (!best) return null
const priceText = (best as cheerio.Cheerio<any>).find('.product__price').first().text().trim()
const price = parsePriceText(priceText)
if (price === null) return null
const link = (best as cheerio.Cheerio<any>).closest('a[href]').attr('href')
return { merchant: 'TopAchat', price, url: link ? new URL(link, url).toString() : url }
}
/**
* Searches Materiel.net by simulating the real search form submission in a
* headless browser. A plain GET on /recherche/?query=... always returns an
* empty "no results" template — the search only actually runs client-side
* (Stimulus/Turbo), so this is one of three merchants where a real browser
* is required rather than a plain HTTP fetch.
*
* When there's a single match, Materiel.net redirects straight to the
* product page. Otherwise it lists cards matched by model token.
*/
async function fetchMaterielNetPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
return withPage(async (page) => {
await page.goto('https://www.materiel.net/', { waitUntil: 'networkidle', timeout: 20000 })
await page.fill('#search_search_text', query)
await page.click('button[type="submit"]')
await page.waitForLoadState('networkidle', { timeout: 20000 })
// Case 1: redirected straight to a single product page (URL pattern
// distinguishes this from Case 2 — a list page also has
// .o-product__price elements, once per card, so checking for its mere
// presence isn't enough to tell the two cases apart).
if (/\/produit\//.test(page.url())) {
const title = await page.title()
if (!modelToken || labelMatchesModelToken(title, modelToken)) {
const text = await page
.locator('.o-product__price')
.first()
.innerText()
.catch(() => '')
const price = parseFlattenedFragmentedPriceText(text) ?? parsePriceText(text)
if (price !== null) {
return { merchant: 'Materiel.net' as const, price, url: page.url() }
}
}
return null
}
// Case 2: a list of results.
const cards = page.locator('li.c-products-list__item[data-id]')
const count = await cards.count()
for (let i = 0; i < count; i++) {
const card = cards.nth(i)
const label = await card
.locator('.c-product__meta h2.c-product__title')
.innerText()
.catch(() => '')
if (modelToken && !labelMatchesModelToken(label, modelToken)) continue
const priceText = await card
.locator('.o-product__price')
.first()
.innerText()
.catch(() => '')
const price = parseFlattenedFragmentedPriceText(priceText) ?? parsePriceText(priceText)
if (price === null) continue
const link = await card
.locator('.c-product__meta a.c-product__link')
.first()
.getAttribute('href')
return { merchant: 'Materiel.net' as const, price, url: link ?? page.url() }
}
return null
})
}
/**
* Searches Cdiscount by product name via a headless browser. Cdiscount's
* search-results page returns an empty body to plain HTTP fetches (no
* error, just nothing), which is consistent with the listing being
* rendered client-side and/or gated behind bot-detection that only lets
* real browsers through.
*/
async function fetchCdiscountPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
const url = `https://www.cdiscount.com/search/10/${encodeURIComponent(query)}.html`
return withPage(async (page) => {
await page.goto(url, { waitUntil: 'networkidle', timeout: 20000 })
const cards = page.locator('div[data-e2e="offer-wrapper"]')
const count = await cards.count()
for (let i = 0; i < count; i++) {
const card = cards.nth(i)
const label = await card
.locator('[data-e2e="lplr-title"]')
.innerText()
.catch(() => '')
if (modelToken && !labelMatchesModelToken(label, modelToken)) continue
const priceText = await card
.locator('[data-e2e="lplr-price"]')
.first()
.innerText()
.catch(() => '')
const price = parsePriceText(priceText)
if (price === null) continue
const link = await card.locator('a').first().getAttribute('href')
const offerUrl = link ? new URL(link, 'https://www.cdiscount.com').toString() : url
return { merchant: 'Cdiscount' as const, price, url: offerUrl }
}
return null
})
}
/**
* Searches PCComponentes.com by product name via a headless browser
* (blocked with a plain HTTP 403 otherwise). Product cards expose clean,
* pre-parsed data on the <a> element itself (data-product-price as a plain
* number, data-product-category to distinguish standalone components from
* full "Sobremesa" pre-built PCs), so full-PC listings that happen to
* mention the same CPU are excluded rather than misreported as its price.
*/
async function fetchPcComponentesPrice(productName: string): Promise<MerchantOffer | null> {
const query = shortSearchQuery(productName)
const modelToken = extractModelToken(productName)
const url = `https://www.pccomponentes.com/buscar/?query=${encodeURIComponent(query)}`
return withPage(async (page) => {
await page.goto(url, { waitUntil: 'networkidle', timeout: 20000 })
const cards = page.locator('#search-results-product-grid > a[data-product-id]')
const count = await cards.count()
for (let i = 0; i < count; i++) {
const card = cards.nth(i)
const category = await card.getAttribute('data-product-category')
if (category === 'Sobremesa') continue // full pre-built PC, not the component itself
const name = (await card.getAttribute('data-product-name')) ?? ''
if (modelToken && !labelMatchesModelToken(name, modelToken)) continue
const priceAttr = await card.getAttribute('data-product-price')
const price = priceAttr ? Number.parseFloat(priceAttr) : null
if (price === null || Number.isNaN(price)) continue
const href = await card.getAttribute('href')
const offerUrl = href ? new URL(href, 'https://www.pccomponentes.com').toString() : url
return { merchant: 'PCComponentes' as const, price, url: offerUrl }
}
return null
})
}
/**
* Fetches offers from every supported merchant in parallel and returns
* them sorted by ascending price. Merchants that fail (blocked, no match,
* network error) are silently skipped rather than failing the whole
* comparison.
*/
async function findCheapestOffers(
productName: string,
amazonUrl: string | null
): Promise<MerchantOffer[]> {
const lookups: Promise<MerchantOffer | null>[] = [
amazonUrl ? fetchAmazonPrice(amazonUrl) : Promise.resolve(null),
fetchLdlcPrice(productName),
fetchGrosbillPrice(productName),
fetchTopAchatPrice(productName),
fetchMaterielNetPrice(productName),
fetchCdiscountPrice(productName),
fetchPcComponentesPrice(productName),
]
const results = await Promise.allSettled(lookups)
const offers = results
.filter(
(result): result is PromiseFulfilledResult<MerchantOffer | null> =>
result.status === 'fulfilled'
)
.map((result) => result.value)
.filter((offer): offer is MerchantOffer => offer !== null)
return offers.sort((a, b) => a.price - b.price)
}
export {
findCheapestOffers,
fetchAmazonPrice,
fetchLdlcPrice,
fetchGrosbillPrice,
fetchTopAchatPrice,
fetchMaterielNetPrice,
fetchCdiscountPrice,
fetchPcComponentesPrice,
extractModelToken,
labelMatchesModelToken,
}
+451
View File
@@ -0,0 +1,451 @@
import * as cheerio from 'cheerio'
import { DateTime } from 'luxon'
import logger from '@adonisjs/core/services/logger'
import ComponentType from '#models/component_type'
import ScrapedPart from '#models/scraped_part'
const SOURCE = 'pc-builder.io'
const BASE_URL = 'https://fr.pc-builder.io'
const USER_AGENT = 'Mozilla/5.0 (compatible; PcBuilderPartFinder/1.0)'
const PAGE_SIZE = 100
/**
* Default freshness window for the scraped_parts cache. A category is
* re-scraped automatically the first time it is searched after this delay
* has elapsed since its last successful scrape.
*/
const CACHE_TTL_HOURS = 24
/**
* How many products to fetch when a search triggers an automatic refresh
* (see ensureFreshCategory). Kept small since this runs synchronously
* inside a user-facing HTTP request — a full deep scrape (thousands of
* products for some categories) belongs in the `scrape:parts` command
* instead, not in the request/response cycle of a search.
*/
const QUICK_REFRESH_MAX_PRODUCTS = 200
/**
* Maps our own ComponentType names to pc-builder.io's category slug,
* numeric category_id (required by the paginated /products/data endpoint,
* found in each category page's inline DataTables config), and how many
* products `node ace scrape:parts` should fetch for that category.
*
* maxProducts is chosen per category based on how much price variance
* matters there and how large the catalog is: GPUs get close-to-full
* coverage since price differences between models are large and users
* specifically look for cheaper alternatives (see the RTX 5070 case that
* prompted this), while smaller/less price-sensitive categories use a
* lighter cap to keep the full scrape run in a reasonable time.
*
* Note: pc-builder.io only has a single "storage" category covering both
* SSDs and HDDs (no separate slugs exist), so it is only mapped to our
* "SSD" type to avoid two types racing to own the same scraped rows
* (source_ref is unique per source, independent of type_id).
*/
const CATEGORY_CONFIG: Record<string, { slug: string; categoryId: string; maxProducts: number }> = {
'Processor': { slug: 'cpu', categoryId: '1', maxProducts: 500 },
'Motherboard': { slug: 'motherboard', categoryId: '3', maxProducts: 500 },
'Graphic Card': { slug: 'gpu', categoryId: '6', maxProducts: 6000 },
'Memory': { slug: 'ram', categoryId: '4', maxProducts: 800 },
'SSD': { slug: 'storage', categoryId: '5', maxProducts: 800 },
'Cooler': { slug: 'cpu-cooler', categoryId: '2', maxProducts: 400 },
'Power Supply': { slug: 'power-supply', categoryId: '8', maxProducts: 400 },
'Case': { slug: 'case', categoryId: '7', maxProducts: 400 },
'Fan': { slug: 'case-fan', categoryId: '20', maxProducts: 300 },
'WiFi-Card': { slug: 'wireless-network-adapter', categoryId: '13', maxProducts: 400 },
}
/** Backwards-compatible slug lookup, kept for anything relying on it. */
const CATEGORY_SLUGS: Record<string, string> = Object.fromEntries(
Object.entries(CATEGORY_CONFIG).map(([name, config]) => [name, config.slug])
)
interface ScrapedRow {
name: string
/**
* Null when pc-builder.io has no fixed offer for this part (common for
* very recent products — it only links to a generic Amazon search,
* e.g. "https://amazon.fr/s?k=..." instead of a priced "/dp/{ASIN}"
* page). Such parts are still kept (not skipped) so they remain
* searchable and importable; ScrapedPartsController#compare lets the
* user resolve — and persist — a real price for them on demand from
* the other supported merchants.
*/
price: number | null
imageUrl: string | null
/**
* The actual merchant page where this part can be bought (currently
* always an Amazon.fr link, since that's the only "buy" link pc-builder.io
* exposes on its category pages) — either a direct priced product page
* or a generic search results page when no price is known. This is
* what gets stored as ScrapedPart.sourceUrl and later copied into
* Component.purchaseUrl — it intentionally is NOT the pc-builder.io
* product page, which isn't a place a user can actually purchase
* anything from.
*/
purchaseUrl: string
sourceRef: string
}
/**
* Loads a category page once to grab the CSRF token and session cookie
* needed to call the paginated /products/data endpoint, which is what
* powers the client-side DataTables widget on every category page.
*/
async function getSessionAndToken(slug: string): Promise<{ cookies: string; token: string }> {
const response = await fetch(`${BASE_URL}/${slug}`, {
headers: { 'User-Agent': USER_AGENT },
})
if (!response.ok) {
throw new Error(`Failed to load ${slug} category page: HTTP ${response.status}`)
}
const html = await response.text()
const setCookieHeaders = response.headers.getSetCookie?.() ?? []
const cookies = setCookieHeaders.map((cookie) => cookie.split(';')[0]).join('; ')
const tokenMatch = html.match(/"_token":\s*"([^"]+)"/)
if (!tokenMatch) {
throw new Error(`Could not find CSRF token on ${slug} category page`)
}
return { cookies, token: tokenMatch[1] }
}
/**
* Parses the "Name" / "Add to Build" cell HTML fragments returned by
* /products/data into a single row, using the exact same markup structure
* as the plain category page (both come from the same Blade partials), so
* the same selectors apply.
*/
function parseRow(nameHtml: string, addToBuildHtml: string): ScrapedRow | null {
const $name = cheerio.load(nameHtml)
const nameLink = $name('a').first()
const name = nameLink.text().trim()
const productPageUrl = nameLink.attr('href')
const img = $name('img.product-image')
const imageUrl = img.attr('data-src') || img.attr('src') || null
const $btn = cheerio.load(addToBuildHtml)
const priceText = $btn('.price-box p').first().text().trim()
const sourceRef = $btn('a.add-to-build').attr('data-id')
const amazonUrl = $btn('a.btn-default').attr('href')
if (!name || !productPageUrl || !sourceRef) {
// Only truly unusable rows (missing name/link/id) are skipped. A
// missing price is not disqualifying — see the ScrapedRow.price doc.
return null
}
let price: number | null = null
if (priceText) {
const parsed = Number.parseFloat(priceText.replace(/[^\d.,]/g, '').replace(',', '.'))
price = Number.isNaN(parsed) ? null : parsed
}
return { name, price, imageUrl, purchaseUrl: amazonUrl || productPageUrl, sourceRef }
}
/**
* Fetches one page of `length` products starting at `start` from
* pc-builder.io's DataTables-backed /products/data endpoint. This is a
* plain HTTP POST (no headless browser needed) once the CSRF
* token/session cookie from getSessionAndToken() are supplied — see
* app/services/part_scraper_service.ts git history / PR description for
* the reverse-engineered request shape (field names matter exactly:
* columns[].data must be "Name"/"Add to Build", not numeric indices).
*/
async function fetchProductsPage(
categoryId: string,
cookies: string,
token: string,
start: number,
length: number
): Promise<{ rows: ScrapedRow[]; recordsTotal: number }> {
const body = new URLSearchParams({
'_token': token,
'draw': '1',
'start': String(start),
'length': String(length),
'compatibility_filter': 'false',
'filter_status': '',
'category_id': categoryId,
'compatible_with': '',
'checkbox_filters': '[]',
'range_filters': '[]',
'search_query': '',
'columns[0][data]': 'Name',
'columns[0][name]': 'Name',
'columns[0][searchable]': 'true',
'columns[0][orderable]': 'false',
'columns[0][search][value]': '',
'columns[0][search][regex]': 'false',
'columns[1][data]': 'Add to Build',
'columns[1][name]': 'Add to Build',
'columns[1][searchable]': 'true',
'columns[1][orderable]': 'false',
'columns[1][search][value]': '',
'columns[1][search][regex]': 'false',
'search[value]': '',
'search[regex]': 'false',
})
const response = await fetch(`${BASE_URL}/products/data`, {
method: 'POST',
headers: {
'User-Agent': USER_AGENT,
'X-Requested-With': 'XMLHttpRequest',
'Content-Type': 'application/x-www-form-urlencoded',
'Cookie': cookies,
},
body,
})
if (!response.ok) {
throw new Error(`products/data request failed: HTTP ${response.status}`)
}
const json = (await response.json()) as {
recordsTotal: number
data: { 'Name': string; 'Add to Build': string }[]
}
const rows = json.data
.map((entry) => parseRow(entry.Name, entry['Add to Build']))
.filter((row): row is ScrapedRow => row !== null)
return { rows, recordsTotal: json.recordsTotal }
}
/**
* Fetches up to `maxProducts` products for a category by paginating
* through pc-builder.io's /products/data endpoint PAGE_SIZE at a time,
* stopping early once the category's real total is exhausted. A short
* delay between pages avoids hammering the site with a burst of requests
* back-to-back for the larger categories (e.g. GPU's ~60 pages).
*/
async function fetchCategoryProducts(
slug: string,
categoryId: string,
maxProducts: number
): Promise<ScrapedRow[]> {
const { cookies, token } = await getSessionAndToken(slug)
const rows: ScrapedRow[] = []
let start = 0
let recordsTotal = maxProducts
while (start < maxProducts && start < recordsTotal) {
const length = Math.min(PAGE_SIZE, maxProducts - start)
const page = await fetchProductsPage(categoryId, cookies, token, start, length)
rows.push(...page.rows)
recordsTotal = page.recordsTotal
start += length
if (start < maxProducts && start < recordsTotal) {
await new Promise((resolve) => setTimeout(resolve, 150))
}
}
return rows
}
/**
* Decides whether a catalog refresh row should overwrite an existing
* ScrapedPart's price fields, or leave them untouched.
*
* Only a price that was actually *verified* via a merchant comparison
* (priceSource set to a merchant name by updatePriceIfCheaper, e.g.
* "LDLC") is protected from being overwritten by a plain catalog refresh
* — that price was confirmed live on the merchant's own page.
*
* A price with priceSource === null just came from a *previous*
* pc-builder.io catalog scrape, with no independent verification.
* pc-builder.io's own listings change constantly — a GPU can have a fixed
* Amazon price today and only a generic "search on Amazon" link (no
* price) tomorrow, if the ASIN it referenced runs out of stock. Treating
* that old, unverified pc-builder.io price as untouchable would freeze it
* forever even after the site itself removed it, which is exactly the
* bug this function fixes (a €431.17 price kept showing for a GPU
* pc-builder.io no longer prices at all).
*/
function shouldKeepExistingPrice(
existing: { price: number | null; priceSource: string | null } | null,
incomingPrice: number | null
): boolean {
if (!existing || existing.price === null || existing.priceSource === null) return false
return incomingPrice === null || existing.price <= incomingPrice
}
/**
* Scrapes a single component type's mapped category and upserts the
* resulting rows into the scraped_parts cache table.
*
* @param maxProductsOverride Caps how many products are fetched for this
* run, regardless of the category's configured maxProducts. Used by
* ensureFreshCategory to keep an on-demand refresh fast (a handful of
* pages) while `node ace scrape:parts` uses the full configured depth.
*/
async function scrapeComponentType(
componentType: ComponentType,
maxProductsOverride?: number
): Promise<number> {
const config = CATEGORY_CONFIG[componentType.name]
if (!config) {
logger.warn({ type: componentType.name }, 'No pc-builder.io category mapped, skipping')
return 0
}
const maxProducts = Math.min(maxProductsOverride ?? config.maxProducts, config.maxProducts)
const rows = await fetchCategoryProducts(config.slug, config.categoryId, maxProducts)
const scrapedAt = DateTime.now()
for (const row of rows) {
const existing = await ScrapedPart.query()
.where('source', SOURCE)
.andWhere('sourceRef', row.sourceRef)
.first()
const keepExistingPrice = shouldKeepExistingPrice(existing, row.price)
await ScrapedPart.updateOrCreate(
{ source: SOURCE, sourceRef: row.sourceRef },
{
typeId: componentType.id,
name: row.name,
imageUrl: row.imageUrl,
scrapedAt,
...(keepExistingPrice
? {}
: {
price: row.price,
sourceUrl: row.purchaseUrl,
priceSource: row.price !== null ? SOURCE : null,
priceCheckedAt: row.price !== null ? scrapedAt : null,
}),
}
)
}
return rows.length
}
/**
* Scrapes every mapped ComponentType category from pc-builder.io and
* refreshes the scraped_parts cache, using each category's full
* configured depth (see CATEGORY_CONFIG). Returns a per-type summary.
* This can take a few minutes overall (GPU alone fetches up to 6000
* products across ~60 requests) — intended to be run manually or via a
* scheduled job, not inside a request.
*/
async function scrapeAllCategories(): Promise<Record<string, number>> {
const componentTypes = await ComponentType.all()
const summary: Record<string, number> = {}
for (const componentType of componentTypes) {
if (!CATEGORY_CONFIG[componentType.name]) continue
try {
summary[componentType.name] = await scrapeComponentType(componentType)
} catch (error) {
logger.error({ type: componentType.name, err: error }, 'Failed to scrape category')
summary[componentType.name] = 0
}
}
return summary
}
/**
* Returns true if the given ComponentType's category has never been
* scraped, or was last scraped more than CACHE_TTL_HOURS ago.
*/
async function isCategoryStale(componentType: ComponentType): Promise<boolean> {
const mostRecent = await ScrapedPart.query()
.where('source', SOURCE)
.andWhere('typeId', componentType.id)
.orderBy('scrapedAt', 'desc')
.first()
if (!mostRecent) return true
return mostRecent.scrapedAt < DateTime.now().minus({ hours: CACHE_TTL_HOURS })
}
/**
* Ensures the cache for a given ComponentType's mapped category is fresh
* (scraped within the last CACHE_TTL_HOURS), re-scraping it synchronously
* if it's missing or stale. No-op for unmapped types. Scraping errors are
* logged but not thrown, so a search still falls back to whatever is
* already cached.
*
* Only fetches QUICK_REFRESH_MAX_PRODUCTS products (not the category's
* full configured depth), since this runs inline in a user's search
* request — run `node ace scrape:parts` for full-depth coverage.
*/
async function ensureFreshCategory(componentType: ComponentType): Promise<void> {
if (!CATEGORY_CONFIG[componentType.name]) return
if (!(await isCategoryStale(componentType))) return
try {
await scrapeComponentType(componentType, QUICK_REFRESH_MAX_PRODUCTS)
} catch (error) {
logger.error({ type: componentType.name, err: error }, 'Failed to refresh stale category cache')
}
}
/**
* Persists `offer` as a part's new price/source if it's cheaper than
* whatever is currently stored (or nothing was stored yet), so the "best
* known price" keeps improving as users compare prices (see
* ScrapedPartsController#compare) instead of resetting to a possibly
* stale/missing pc-builder.io price on the next catalog refresh —
* scrapeComponentType already refuses to overwrite a price with a worse
* one using this same "keep whichever is lower" rule.
*/
async function updatePriceIfCheaper(
part: ScrapedPart,
offer: { merchant: string; price: number; url: string }
): Promise<void> {
if (part.price !== null && part.price <= offer.price) return
part.price = offer.price
part.sourceUrl = offer.url
part.priceSource = offer.merchant
part.priceCheckedAt = DateTime.now()
await part.save()
}
/**
* Clears a part's price when a live comparison found no matching offer on
* any supported merchant (see ScrapedPartsController#compare), regardless
* of where the price previously came from (pc-builder.io catalog or an
* earlier verified merchant match). The pc-builder.io catalog price this
* part started with is frequently stale (see shouldKeepExistingPrice's
* doc for a confirmed example: pc-builder.io showed €128 for a CPU
* actually selling for €66.89) — once a user explicitly asks to verify a
* price and no merchant confirms it anymore, keeping the old number
* around is misleading, so it's dropped to null instead of silently
* falling back to "using catalog price €X".
*/
async function clearPriceIfNoOffers(part: ScrapedPart): Promise<void> {
if (part.price === null) return
part.price = null
part.priceSource = null
part.priceCheckedAt = DateTime.now()
await part.save()
}
export {
scrapeAllCategories,
scrapeComponentType,
ensureFreshCategory,
isCategoryStale,
updatePriceIfCheaper,
clearPriceIfNoOffers,
shouldKeepExistingPrice,
CATEGORY_CONFIG,
CATEGORY_SLUGS,
CACHE_TTL_HOURS,
QUICK_REFRESH_MAX_PRODUCTS,
}
@@ -0,0 +1,30 @@
import type Computer from '#models/computer'
/**
* Transforms a Computer into the shape exposed on the public share page.
* Unlike the regular serialize(), this never leaks the owner id, and only
* includes sellPrice/cost when the owner explicitly enabled that on the
* share settings.
*/
export default class ComputerPublicTransformer {
static transform(computer: typeof Computer.prototype) {
return {
id: computer.id,
title: computer.title,
gpuScore: computer.gpuScore,
cpuScore: computer.cpuScore,
globalScore: computer.globalScore,
state: { name: computer.state.name },
sellPrice: computer.shareShowSellPrice ? computer.sellPrice : null,
cost: computer.shareShowCost ? computer.cost : null,
pictures: computer.pictures
.map((picture) => ({ link: picture.link, order: picture.order }))
.sort((a, b) => a.order - b.order),
components: computer.components.map((component) => ({
id: component.id,
name: component.name,
type: { name: component.type.name },
})),
}
}
}
+7
View File
@@ -6,6 +6,13 @@ export const storeComponentValidator = vine.compile(
price: vine.number().decimal([0, 2]).min(0),
typeId: vine.number().min(1),
purchaseDate: vine.string().regex(/^\d{4}-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])$/),
purchaseUrl: vine
.string()
.trim()
.url({ require_protocol: true })
.maxLength(2048)
.nullable()
.optional(),
quantity: vine.number().min(1).withoutDecimals(),
})
)
+7
View File
@@ -6,5 +6,12 @@ export const updateComponentValidator = vine.compile(
price: vine.number().decimal([0, 2]).min(0),
typeId: vine.number().min(1),
purchaseDate: vine.string().regex(/^\d{4}-(0[1-9]|1[012])-(0[1-9]|[12][0-9]|3[01])$/),
purchaseUrl: vine
.string()
.trim()
.url({ require_protocol: true })
.maxLength(2048)
.nullable()
.optional(),
})
)
+9
View File
@@ -0,0 +1,9 @@
import vine from '@vinejs/vine'
export const updateShareSettingsValidator = vine.compile(
vine.object({
enabled: vine.boolean(),
showSellPrice: vine.boolean(),
showCost: vine.boolean(),
})
)
+35
View File
@@ -0,0 +1,35 @@
import { BaseCommand } from '@adonisjs/core/ace'
import type { CommandOptions } from '@adonisjs/core/types/ace'
export default class ScrapeParts extends BaseCommand {
static commandName = 'scrape:parts'
static description = 'Refresh the new-parts cache by scraping pc-builder.io category pages'
static options: CommandOptions = {
startApp: true,
}
async run() {
const { scrapeAllCategories } = await import('#services/part_scraper_service')
const { seedManualParts } = await import('#services/manual_part_seeder_service')
this.logger.info('Scraping pc-builder.io categories...')
const summary = await scrapeAllCategories()
let total = 0
for (const [type, count] of Object.entries(summary)) {
this.logger.info(` ${type}: ${count} parts`)
total += count
}
this.logger.success(
`Done. Upserted ${total} parts across ${Object.keys(summary).length} categories.`
)
this.logger.info('Seeding manually-tracked parts not yet in pc-builder.io catalog...')
const manualResult = await seedManualParts()
this.logger.success(
`Manual parts: ${manualResult.inserted} upserted, ${manualResult.skipped} skipped.`
)
}
}
+38 -6
View File
@@ -1,4 +1,18 @@
import { defineConfig } from '@adonisjs/shield'
import app from '@adonisjs/core/services/app'
/**
* In development, Vite's dev server assets (scripts, styles, /@vite/client)
* are proxied through AdonisJS on the same origin, so 'self' already covers
* them — there is no working "@viteDevUrl"/"@viteHmrUrl" CSP keyword in this
* version of @adonisjs/shield (only "@viteUrl" exists, and it resolves to an
* empty string here since assetsUrl is a relative path, not an absolute
* dev-server URL). The one thing that truly lives on a different origin is
* the Vite HMR websocket, which runs on its own port (see VITE_HMR_PORT,
* default 24678) and therefore needs an explicit connect-src entry.
*/
const viteHmrPort = process.env.VITE_HMR_PORT ?? '24678'
const devConnectSrc = app.inDev ? [`ws://localhost:${viteHmrPort}`] : []
const shieldConfig = defineConfig({
/**
@@ -15,12 +29,30 @@ const shieldConfig = defineConfig({
* Per-resource CSP directives.
*/
directives: {
defaultSrc: [`'self'`, '@viteDevUrl'],
connectSrc: [`'self'`, '@viteHmrUrl'],
scriptSrc: [`'self'`, '@viteDevUrl'],
styleSrc: [`'self'`, '@viteDevUrl', `'unsafe-inline'`],
imgSrc: [`'self'`, 'data:'],
fontSrc: [`'self'`, 'data:'],
defaultSrc: [`'self'`],
connectSrc: [`'self'`, ...devConnectSrc],
scriptSrc: [`'self'`],
// Some client-side dependencies (e.g. Vite's HMR client / the Tuyau
// API client) spawn a Web Worker from a `blob:` URL built at
// runtime from already same-origin-loaded code. Without an explicit
// worker-src, browsers fall back to script-src ('self' only), which
// blocks the blob: worker outright — this was observed to break the
// dev HMR connection and leave the Vue app in an inconsistent state
// (lost TuyauProvider context after a failed hot reload).
workerSrc: [`'self'`, 'blob:'],
styleSrc: [`'self'`, `'unsafe-inline'`, 'https://fonts.googleapis.com'],
// The favicon service (used to show a merchant's icon in the price
// comparison UI) is requested from www.google.com but actually
// serves/redirects to Google's gstatic CDN (t0-t3.gstatic.com),
// hence the wildcard.
imgSrc: [
`'self'`,
'data:',
'https://fr.pc-builder.io',
'https://www.google.com',
'https://*.gstatic.com',
],
fontSrc: [`'self'`, 'data:', 'https://fonts.gstatic.com'],
},
/**
@@ -0,0 +1,23 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'computers'
async up() {
this.schema.alterTable(this.tableName, (table) => {
table.string('share_token', 64).nullable().unique()
table.boolean('share_enabled').notNullable().defaultTo(false)
table.boolean('share_show_sell_price').notNullable().defaultTo(false)
table.boolean('share_show_cost').notNullable().defaultTo(false)
})
}
async down() {
this.schema.alterTable(this.tableName, (table) => {
table.dropColumn('share_token')
table.dropColumn('share_enabled')
table.dropColumn('share_show_sell_price')
table.dropColumn('share_show_cost')
})
}
}
@@ -0,0 +1,28 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'scraped_parts'
async up() {
this.schema.createTable(this.tableName, (table) => {
table.increments('id')
table.integer('type_id').unsigned().references('id').inTable('component_types').notNullable()
table.string('name').notNullable()
table.float('price', 8, 2).notNullable()
table.string('image_url').nullable()
table.string('source').notNullable().defaultTo('pc-builder.io')
table.string('source_url').notNullable()
table.string('source_ref').notNullable()
table.timestamp('scraped_at').notNullable()
table.timestamp('created_at')
table.timestamp('updated_at')
table.unique(['source', 'source_ref'])
table.index(['type_id'])
})
}
async down() {
this.schema.dropTable(this.tableName)
}
}
@@ -0,0 +1,17 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'components'
async up() {
this.schema.alterTable(this.tableName, (table) => {
table.string('purchase_url').nullable()
})
}
async down() {
this.schema.alterTable(this.tableName, (table) => {
table.dropColumn('purchase_url')
})
}
}
@@ -0,0 +1,32 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
protected tableName = 'scraped_parts'
async up() {
this.schema.alterTable(this.tableName, (table) => {
// Not every pc-builder.io listing has a fixed price (recent GPUs in
// particular often only have a generic "search on Amazon" link with
// no resolved offer) — these are now kept instead of being skipped,
// so price must be nullable.
table.float('price', 8, 2).nullable().alter()
// Tracks where the currently stored `price`/`source_url` came from:
// "pc-builder.io" for the original catalog price, or one of the
// merchant names from merchant_price_service.ts (Amazon, LDLC, ...)
// once a live price comparison finds (and persists) a cheaper offer.
// Null means no price has ever been resolved for this part.
table.string('price_source').nullable()
table.timestamp('price_checked_at').nullable()
})
}
async down() {
this.schema.alterTable(this.tableName, (table) => {
table.dropColumn('price_source')
table.dropColumn('price_checked_at')
})
this.schema.alterTable(this.tableName, (table) => {
table.float('price', 8, 2).notNullable().alter()
})
}
}
@@ -0,0 +1,27 @@
import { BaseSchema } from '@adonisjs/lucid/schema'
export default class extends BaseSchema {
async up() {
// Some merchants (Grosbill in particular) return long tracking
// redirect URLs with encoded analytics query params well past 255
// characters — varchar(255) was truncating/rejecting inserts. Widen
// both scraped_parts.source_url (where these URLs are first stored)
// and components.purchase_url (where they end up when a scraped part
// gets imported into a Component — see components_controller.ts).
this.schema.alterTable('scraped_parts', (table) => {
table.text('source_url').notNullable().alter()
})
this.schema.alterTable('components', (table) => {
table.text('purchase_url').nullable().alter()
})
}
async down() {
this.schema.alterTable('scraped_parts', (table) => {
table.string('source_url').notNullable().alter()
})
this.schema.alterTable('components', (table) => {
table.string('purchase_url').nullable().alter()
})
}
}
+43 -2
View File
@@ -19,7 +19,7 @@ export class ComponentTypeSchema extends BaseModel {
}
export class ComponentSchema extends BaseModel {
static $columns = ['computerId', 'createdAt', 'id', 'name', 'price', 'purchaseDate', 'typeId', 'updatedAt', 'userId'] as const
static $columns = ['computerId', 'createdAt', 'id', 'name', 'price', 'purchaseDate', 'purchaseUrl', 'typeId', 'updatedAt', 'userId'] as const
$columns = ComponentSchema.$columns
@column()
declare computerId: number | null
@@ -34,6 +34,8 @@ export class ComponentSchema extends BaseModel {
@column.date()
declare purchaseDate: DateTime
@column()
declare purchaseUrl: string | null
@column()
declare typeId: number | null
@column.dateTime({ autoCreate: true, autoUpdate: true })
declare updatedAt: DateTime | null
@@ -68,7 +70,7 @@ export class ComputerStateSchema extends BaseModel {
}
export class ComputerSchema extends BaseModel {
static $columns = ['cpuScore', 'createdAt', 'globalScore', 'gpuScore', 'id', 'sellPrice', 'stateId', 'title', 'updatedAt', 'userId'] as const
static $columns = ['cpuScore', 'createdAt', 'globalScore', 'gpuScore', 'id', 'sellPrice', 'shareEnabled', 'shareShowCost', 'shareShowSellPrice', 'shareToken', 'stateId', 'title', 'updatedAt', 'userId'] as const
$columns = ComputerSchema.$columns
@column()
declare cpuScore: number
@@ -83,6 +85,14 @@ export class ComputerSchema extends BaseModel {
@column()
declare sellPrice: number | null
@column()
declare shareEnabled: boolean
@column()
declare shareShowCost: boolean
@column()
declare shareShowSellPrice: boolean
@column()
declare shareToken: string | null
@column()
declare stateId: number
@column()
declare title: string
@@ -92,6 +102,37 @@ export class ComputerSchema extends BaseModel {
declare userId: number | null
}
export class ScrapedPartSchema extends BaseModel {
static $columns = ['createdAt', 'id', 'imageUrl', 'name', 'price', 'priceCheckedAt', 'priceSource', 'scrapedAt', 'source', 'sourceRef', 'sourceUrl', 'typeId', 'updatedAt'] as const
$columns = ScrapedPartSchema.$columns
@column.dateTime({ autoCreate: true })
declare createdAt: DateTime | null
@column({ isPrimary: true })
declare id: number
@column()
declare imageUrl: string | null
@column()
declare name: string
@column()
declare price: number | null
@column.dateTime()
declare priceCheckedAt: DateTime | null
@column()
declare priceSource: string | null
@column.dateTime()
declare scrapedAt: DateTime
@column()
declare source: string
@column()
declare sourceRef: string
@column()
declare sourceUrl: string
@column()
declare typeId: number
@column.dateTime({ autoCreate: true, autoUpdate: true })
declare updatedAt: DateTime | null
}
export class UserSchema extends BaseModel {
static $columns = ['createdAt', 'email', 'id', 'name', 'password', 'updatedAt'] as const
$columns = UserSchema.$columns
+34 -7
View File
@@ -3,17 +3,32 @@ import { reactive } from 'vue'
import { Link } from '@adonisjs/inertia/vue'
import { useForm } from '@inertiajs/vue3'
import CustomSelect from '~/pages/widgets/CustomSelect.vue'
import type { ComponentType } from '~/pages/types/UITypes'
import NewPartSearch from '~/pages/widgets/NewPartSearch.vue'
import type { ComponentType, MerchantOffer, ScrapedPart } from '~/pages/types/UITypes'
defineProps<{ types: ComponentType[]; errors?: Record<string, string> }>()
const form = reactive({
name: '',
price: 0,
typeId: null,
typeId: null as number | null,
purchaseDate: new Date().toISOString().split('T')[0],
purchaseUrl: '',
quantity: 1,
})
function importPart(part: ScrapedPart, cheapestOffer: MerchantOffer | null) {
form.name = part.name
// Prefer the live price/link found by comparing merchants (Amazon, LDLC,
// Grosbill, TopAchat, ...) when available, since pc-builder.io's catalog
// price can be stale (or simply unknown — see ScrapedPart.price doc);
// fall back to the catalog price/link, or leave the price field as-is
// for the user to fill in manually if neither is known.
form.price = cheapestOffer?.price ?? part.price ?? form.price
form.typeId = part.type.id
form.purchaseDate = new Date().toISOString().split('T')[0]
form.purchaseUrl = cheapestOffer?.url ?? part.sourceUrl
}
function submitForm() {
const inertiaForm = useForm(form)
inertiaForm.post('/components')
@@ -23,6 +38,9 @@ function submitForm() {
<template>
<div class="max-w-2xl mx-auto p-8 bg-white shadow-lg rounded-lg">
<h1 class="text-2xl font-semibold mb-8 text-gray-800">Create New Component</h1>
<NewPartSearch class="mb-8" @import="importPart" />
<form @submit.prevent="submitForm" class="space-y-6">
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="name">Name</label>
@@ -48,11 +66,7 @@ function submitForm() {
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="type">Type</label>
<CustomSelect
v-model="form.typeId"
:options="types"
placeholder="Choose a type"
/>
<CustomSelect v-model="form.typeId" :options="types" placeholder="Choose a type" />
</div>
<div>
@@ -67,6 +81,19 @@ function submitForm() {
/>
</div>
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="purchaseUrl"
>Purchase Site (optional, filled automatically when importing a new part)</label
>
<input
name="purchaseUrl"
v-model="form.purchaseUrl"
class="w-full mt-2 border-gray-200 focus:border-indigo-600 focus:ring focus:ring-indigo-500/40 px-3 py-2 border rounded text-gray-700"
type="url"
placeholder="https://..."
/>
</div>
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="quantity">Quantity</label>
<input
+15 -5
View File
@@ -18,6 +18,7 @@ const form = reactive({
price: component.price,
typeId: component.type.id,
purchaseDate: component.purchaseDate,
purchaseUrl: component.purchaseUrl ?? '',
})
</script>
@@ -48,11 +49,7 @@ const form = reactive({
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="type">Type</label>
<CustomSelect
v-model="form.typeId"
:options="types"
placeholder="Choose a type"
/>
<CustomSelect v-model="form.typeId" :options="types" placeholder="Choose a type" />
</div>
<div>
@@ -67,6 +64,19 @@ const form = reactive({
/>
</div>
<div>
<label class="block text-gray-700 text-sm font-bold mb-2" for="purchaseUrl"
>Purchase Site (optional)</label
>
<input
name="purchaseUrl"
v-model="form.purchaseUrl"
class="w-full mt-2 border-gray-200 focus:border-indigo-600 focus:ring focus:ring-indigo-500/40 px-3 py-2 border rounded text-gray-700"
type="url"
placeholder="https://..."
/>
</div>
<Link
:href="`/components/${form.id}`"
method="put"
+12
View File
@@ -36,6 +36,18 @@ const props = defineProps<{ component: Component }>()
</div>
</div>
<div v-if="props.component.purchaseUrl" class="mb-6">
<label class="block text-gray-700 text-sm font-bold mb-2">Purchase Site</label>
<a
:href="props.component.purchaseUrl"
target="_blank"
rel="noopener noreferrer"
class="block px-3 py-2 border rounded text-indigo-600 hover:text-indigo-800 hover:underline truncate"
>
{{ props.component.purchaseUrl }}
</a>
</div>
<Link
:href="`/components/${props.component.id}/edit`"
type="button"
+22 -3
View File
@@ -1,15 +1,34 @@
<script setup lang="ts">
import { ref } from 'vue'
import { Link } from '@adonisjs/inertia/vue'
import type { Computer } from '~/pages/types/UITypes'
import ComputerShareModal from '~/pages/widgets/ComputerShareModal.vue'
const { computer } = defineProps<{ computer: Computer }>()
const showShareModal = ref(false)
</script>
<template>
<div class="max-w-4xl mx-auto p-4 md:p-8 bg-white shadow-lg rounded-lg">
<h1 class="text-xl md:text-2xl font-semibold mb-4 md:mb-8 text-gray-800">
{{ computer.title }}
</h1>
<div class="flex items-center justify-between mb-4 md:mb-8">
<h1 class="text-xl md:text-2xl font-semibold text-gray-800">
{{ computer.title }}
</h1>
<button
type="button"
@click="showShareModal = true"
class="px-4 py-2 bg-gray-200 text-gray-800 rounded hover:bg-gray-300 transition text-sm shrink-0"
>
{{ computer.shareEnabled ? 'Shared' : 'Share' }}
</button>
</div>
<ComputerShareModal
:show="showShareModal"
:computer="computer"
@close="showShareModal = false"
/>
<div class="mt-2">
<label class="block text-gray-700 text-sm font-bold mb-2">Components</label>
+92
View File
@@ -0,0 +1,92 @@
<script setup lang="ts">
import EmptyLayout from '~/pages/layout/emptyLayout.vue'
import { noImagePlaceholder } from '~/assets/images'
import type { PublicComputer } from '~/pages/types/UITypes'
defineOptions({ layout: EmptyLayout })
const { computer } = defineProps<{ computer: PublicComputer }>()
</script>
<template>
<div class="min-h-screen bg-gradient-to-br from-gray-100 to-gray-200 py-8 px-4">
<div class="max-w-3xl mx-auto p-4 md:p-8 bg-white shadow-lg rounded-lg">
<p class="text-xs uppercase tracking-wide text-indigo-600 font-semibold mb-2">Shared build</p>
<h1 class="text-xl md:text-2xl font-semibold mb-6 text-gray-800">
{{ computer.title }}
</h1>
<img
class="w-full max-h-96 object-cover rounded-lg mb-6"
:src="computer.pictures[0]?.link || noImagePlaceholder"
alt="Computer picture"
/>
<div class="mt-2">
<label class="block text-gray-700 text-sm font-bold mb-2">Components</label>
<div class="overflow-x-auto">
<table class="w-full border-collapse">
<thead class="bg-gray-200">
<tr>
<th class="border px-4 py-2 text-left">Name</th>
<th class="border px-4 py-2 text-left">Type</th>
</tr>
</thead>
<tbody>
<tr
v-for="component in computer.components"
:key="component.id"
class="hover:bg-gray-50"
>
<td class="border px-4 py-2">{{ component.name }}</td>
<td class="border px-4 py-2">{{ component.type?.name }}</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="mt-6 flex flex-wrap gap-4">
<div class="flex-1 min-w-[100px]">
<label class="block text-gray-700 text-sm font-bold mb-2">GPU Score</label>
<p class="w-full border-gray-200 px-3 py-2 border rounded text-gray-700">
{{ computer.gpuScore }}
</p>
</div>
<div class="flex-1 min-w-[100px]">
<label class="block text-gray-700 text-sm font-bold mb-2">CPU Score</label>
<p class="w-full border-gray-200 px-3 py-2 border rounded text-gray-700">
{{ computer.cpuScore }}
</p>
</div>
<div class="flex-1 min-w-[100px]">
<label class="block text-gray-700 text-sm font-bold mb-2">Global Score</label>
<p class="w-full border-gray-200 px-3 py-2 border rounded text-gray-700">
{{ computer.globalScore }}
</p>
</div>
</div>
<div class="mt-4 grid grid-cols-1 md:grid-cols-3 gap-4">
<div class="pt-2">
<label class="block text-gray-700 text-sm font-bold mb-2">State</label>
<p class="w-full mt-2 border-gray-200 px-3 py-2.5 border rounded text-gray-700">
{{ computer.state.name }}
</p>
</div>
<div v-if="computer.sellPrice !== null" class="mt-2">
<label class="block text-gray-700 text-sm font-bold mb-2">Sell Price</label>
<p class="w-full mt-2 border-gray-200 px-3 py-2 border rounded text-gray-700">
${{ computer.sellPrice.toFixed(2) }}
</p>
</div>
<div v-if="computer.cost !== null" class="mt-2">
<label class="block text-gray-700 text-sm font-bold mb-2">Cost</label>
<p class="w-full mt-2 border-gray-200 px-3 py-2 border rounded text-gray-700">
${{ computer.cost.toFixed(2) }}
</p>
</div>
</div>
</div>
</div>
</template>
+44
View File
@@ -20,6 +20,23 @@ export type Computer = {
cost: number
pictures: { link: string; order: number }[]
components: Component[]
shareEnabled: boolean
shareShowSellPrice: boolean
shareShowCost: boolean
shareToken?: string | null
}
export type PublicComputer = {
id: number
title: string
gpuScore: number
cpuScore: number
globalScore: number
state: { name: string }
sellPrice: number | null
cost: number | null
pictures: { link: string; order: number }[]
components: { id: number; name: string; type: { name: string } }[]
}
export type Component = {
@@ -28,11 +45,38 @@ export type Component = {
price: string
computerId: number | null
purchaseDate: string
purchaseUrl: string | null
createdAt: string
updatedAt: string
type: ComponentType
}
export type ScrapedPart = {
id: number
name: string
// Null when pc-builder.io has no fixed offer for this part yet (common
// for very recent products). Comparing prices (see NewPartSearch.vue)
// can resolve and persist a real price from another merchant.
price: number | null
imageUrl: string | null
source: string
sourceUrl: string
type: ComponentType
}
export type MerchantOffer = {
merchant:
| 'Amazon'
| 'LDLC'
| 'Grosbill'
| 'TopAchat'
| 'Materiel.net'
| 'Cdiscount'
| 'PCComponentes'
price: number
url: string
}
export interface ServerImage {
id: number
link: string
@@ -0,0 +1,111 @@
<script setup lang="ts">
import { computed, reactive, ref } from 'vue'
import { useForm } from '@inertiajs/vue3'
import type { Computer } from '~/pages/types/UITypes'
const props = defineProps<{
show: boolean
computer: Computer
}>()
const emit = defineEmits(['close'])
const form = reactive({
enabled: props.computer.shareEnabled,
showSellPrice: props.computer.shareShowSellPrice,
showCost: props.computer.shareShowCost,
})
const copied = ref(false)
const shareUrl = computed(() => {
if (!props.computer.shareToken) return ''
return `${window.location.origin}/share/${props.computer.shareToken}`
})
function submit() {
useForm(form).put(`/computers/${props.computer.id}/share`, {
preserveScroll: true,
onSuccess: () => emit('close'),
})
}
function rotateToken() {
useForm({}).post(`/computers/${props.computer.id}/share/rotate`, {
preserveScroll: true,
})
}
async function copyLink() {
await navigator.clipboard.writeText(shareUrl.value)
copied.value = true
setTimeout(() => (copied.value = false), 2000)
}
</script>
<template>
<div v-if="show" class="fixed inset-0 flex items-center justify-center z-50">
<div class="fixed inset-0 bg-black opacity-50" @click="emit('close')"></div>
<div class="bg-white rounded-lg p-6 z-10 w-full max-w-md mx-4">
<h2 class="text-lg font-semibold mb-4 text-gray-800">Share this build</h2>
<label class="flex items-center gap-2 mb-4 cursor-pointer">
<input v-model="form.enabled" type="checkbox" class="w-4 h-4" />
<span class="text-sm font-medium text-gray-700">Enable public link</span>
</label>
<div v-if="form.enabled" class="space-y-3 mb-4 pl-6">
<label class="flex items-center gap-2 cursor-pointer">
<input v-model="form.showSellPrice" type="checkbox" class="w-4 h-4" />
<span class="text-sm text-gray-700">Show sell price</span>
</label>
<label class="flex items-center gap-2 cursor-pointer">
<input v-model="form.showCost" type="checkbox" class="w-4 h-4" />
<span class="text-sm text-gray-700">Show components cost</span>
</label>
</div>
<div v-if="computer.shareEnabled && computer.shareToken" class="mb-4">
<label class="block text-gray-700 text-sm font-bold mb-2">Public link</label>
<div class="flex gap-2">
<input
:value="shareUrl"
readonly
class="flex-1 border-gray-200 px-3 py-2 border rounded text-gray-700 text-sm bg-gray-50"
/>
<button
type="button"
@click="copyLink"
class="px-3 py-2 bg-gray-200 text-gray-700 rounded hover:bg-gray-300 text-sm shrink-0"
>
{{ copied ? 'Copied!' : 'Copy' }}
</button>
</div>
<button
type="button"
@click="rotateToken"
class="mt-2 text-xs text-red-600 hover:text-red-800 underline"
>
Revoke and generate a new link
</button>
</div>
<div class="flex justify-end space-x-2 mt-6">
<button
type="button"
@click="submit"
class="px-4 py-2 bg-indigo-600 text-white rounded hover:bg-indigo-700"
>
Save
</button>
<button
type="button"
@click="emit('close')"
class="px-4 py-2 bg-gray-300 text-gray-700 rounded hover:bg-gray-400"
>
Close
</button>
</div>
</div>
</div>
</template>
+194
View File
@@ -0,0 +1,194 @@
<script setup lang="ts">
import { reactive, ref, watch } from 'vue'
import type { MerchantOffer, ScrapedPart } from '~/pages/types/UITypes'
const emit = defineEmits<{
import: [part: ScrapedPart, offer: MerchantOffer | null]
}>()
function faviconUrl(source: string): string {
return `https://www.google.com/s2/favicons?domain=${encodeURIComponent(source)}&sz=32`
}
const query = ref('')
const results = ref<ScrapedPart[]>([])
const isLoading = ref(false)
const hasSearched = ref(false)
let debounceTimer: ReturnType<typeof setTimeout> | null = null
// Per-part price comparison state, keyed by scraped part id.
const comparing = reactive<Record<number, boolean>>({})
const offers = reactive<Record<number, MerchantOffer[]>>({})
const compareErrors = reactive<Record<number, boolean>>({})
async function search() {
if (!query.value.trim()) {
results.value = []
hasSearched.value = false
return
}
isLoading.value = true
try {
const response = await fetch(`/scraped-parts?q=${encodeURIComponent(query.value.trim())}`, {
headers: { Accept: 'application/json' },
})
if (response.ok) {
results.value = await response.json()
} else {
results.value = []
}
} finally {
isLoading.value = false
hasSearched.value = true
}
}
watch(query, () => {
if (debounceTimer) clearTimeout(debounceTimer)
debounceTimer = setTimeout(search, 400)
})
async function comparePrices(part: ScrapedPart) {
comparing[part.id] = true
compareErrors[part.id] = false
try {
const response = await fetch(`/scraped-parts/compare?id=${part.id}`, {
headers: { Accept: 'application/json' },
})
if (!response.ok) {
compareErrors[part.id] = true
return
}
const data = await response.json()
offers[part.id] = data.offers ?? []
// The backend already persists this exact value server-side (see
// ScrapedPartsController#compare): the cheapest offer's price when
// one was found (updatePriceIfCheaper), or null when no merchant
// confirmed a price at all (clearPriceIfNoOffers) — an unverified
// pc-builder.io catalog price is not kept around just because the
// comparison came back empty. Mirror that here on the locally held
// result so the display (e.g. "€979.95" or "No price yet") reflects
// it immediately, instead of the initial search-result price sticking
// around until the user re-runs the search from scratch.
part.price = data.price ?? null
if (offers[part.id].length === 0) {
compareErrors[part.id] = true
}
} catch {
compareErrors[part.id] = true
} finally {
comparing[part.id] = false
}
}
function cheapestOffer(part: ScrapedPart): MerchantOffer | null {
return offers[part.id]?.[0] ?? null
}
function merchantDomain(merchant: MerchantOffer['merchant']): string {
const domains: Record<MerchantOffer['merchant'], string> = {
'Amazon': 'amazon.fr',
'LDLC': 'ldlc.com',
'Grosbill': 'grosbill.com',
'TopAchat': 'topachat.com',
'Materiel.net': 'materiel.net',
'Cdiscount': 'cdiscount.com',
'PCComponentes': 'pccomponentes.com',
}
return domains[merchant]
}
function importPart(part: ScrapedPart) {
emit('import', part, cheapestOffer(part))
}
</script>
<template>
<div class="border border-gray-200 rounded-lg p-4 bg-gray-50">
<label class="block text-gray-700 text-sm font-bold mb-2">
Search a new part (pc-builder.io)
</label>
<input
v-model="query"
type="text"
placeholder="e.g. Ryzen 5 7600X, RTX 4070..."
class="w-full border-gray-200 focus:border-indigo-600 focus:ring focus:ring-indigo-500/40 px-3 py-2 border rounded text-gray-700"
/>
<p v-if="isLoading" class="text-sm text-gray-500 mt-2">Searching…</p>
<p v-else-if="hasSearched && results.length === 0" class="text-sm text-gray-500 mt-2">
No matching part found.
</p>
<p v-else-if="results.length > 0" class="text-xs text-gray-500 mt-2">
{{ results.length }} result{{ results.length > 1 ? 's' : '' }} — scroll for more
</p>
<ul v-if="results.length > 0" class="mt-1 space-y-2 max-h-96 overflow-y-auto">
<li
v-for="part in results"
:key="part.id"
class="bg-white border border-gray-200 rounded p-2"
>
<div class="flex items-center gap-3">
<img
v-if="part.imageUrl"
:src="part.imageUrl"
alt=""
class="w-10 h-10 object-contain shrink-0"
/>
<div class="flex-1 min-w-0">
<p class="text-sm text-gray-800 truncate" :title="part.name">{{ part.name }}</p>
<p class="text-xs text-gray-500">
{{ part.type.name }}
<span v-if="part.price !== null"> · €{{ part.price.toFixed(2) }}</span>
<span v-else class="text-gray-400"> · No price yet</span>
</p>
</div>
<button
type="button"
class="px-3 py-1 text-sm bg-indigo-600 text-white rounded hover:bg-indigo-700 transition shrink-0"
@click="importPart(part)"
>
Import
</button>
</div>
<div class="mt-2 pl-13 flex items-center gap-2 flex-wrap">
<button
v-if="!offers[part.id]"
type="button"
:disabled="comparing[part.id]"
class="text-xs px-2 py-1 border border-gray-300 rounded text-gray-600 hover:bg-gray-100 transition disabled:opacity-50"
@click="comparePrices(part)"
>
{{ comparing[part.id] ? 'Checking prices…' : 'Compare prices' }}
</button>
<template v-else-if="cheapestOffer(part)">
<img
:src="faviconUrl(merchantDomain(cheapestOffer(part)!.merchant))"
alt=""
class="w-4 h-4 shrink-0"
/>
<a
:href="cheapestOffer(part)!.url"
target="_blank"
rel="noopener noreferrer"
class="text-xs text-indigo-600 hover:underline"
>
{{ cheapestOffer(part)!.merchant }}: €{{ cheapestOffer(part)!.price.toFixed(2) }}
(cheapest)
</a>
</template>
<span v-if="compareErrors[part.id]" class="text-xs text-gray-400">
No price found on any merchant for this part.
</span>
</div>
</li>
</ul>
</div>
</template>
+287 -2
View File
@@ -25,9 +25,11 @@
"@vue/server-renderer": "^3.5.35",
"@vuepic/vue-datepicker": "^13.0.0",
"better-sqlite3": "^12.10.0",
"cheerio": "^1.2.0",
"edge.js": "^6.5.1",
"luxon": "^3.7.2",
"mysql2": "^3.22.4",
"playwright-core": "^1.61.1",
"reflect-metadata": "^0.2.2",
"uuid": "^14.0.1",
"vue": "^3.5.35",
@@ -4158,7 +4160,6 @@
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==",
"dev": true,
"license": "ISC"
},
"node_modules/brace-expansion": {
@@ -4422,6 +4423,48 @@
"node": ">=16"
}
},
"node_modules/cheerio": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/cheerio/-/cheerio-1.2.0.tgz",
"integrity": "sha512-WDrybc/gKFpTYQutKIK6UvfcuxijIZfMfXaYm8NMsPQxSYvf+13fXUJ4rztGGbJcBQ/GF55gvrZ0Bc0bj/mqvg==",
"license": "MIT",
"dependencies": {
"cheerio-select": "^2.1.0",
"dom-serializer": "^2.0.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.2",
"encoding-sniffer": "^0.2.1",
"htmlparser2": "^10.1.0",
"parse5": "^7.3.0",
"parse5-htmlparser2-tree-adapter": "^7.1.0",
"parse5-parser-stream": "^7.1.2",
"undici": "^7.19.0",
"whatwg-mimetype": "^4.0.0"
},
"engines": {
"node": ">=20.18.1"
},
"funding": {
"url": "https://github.com/cheeriojs/cheerio?sponsor=1"
}
},
"node_modules/cheerio-select": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/cheerio-select/-/cheerio-select-2.1.0.tgz",
"integrity": "sha512-9v9kG0LvzrlcungtnJtpGNxY+fzECQKhK4EGJX2vByejiMX84MFNQw4UxPJl3bFbTMw+Dfs37XaIkCwTZfLh4g==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-select": "^5.1.0",
"css-what": "^6.1.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3",
"domutils": "^3.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/chokidar": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/chokidar/-/chokidar-5.0.0.tgz",
@@ -4717,6 +4760,34 @@
"node": ">= 0.8"
}
},
"node_modules/css-select": {
"version": "5.2.2",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz",
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-what": "^6.1.0",
"domhandler": "^5.0.2",
"domutils": "^3.0.1",
"nth-check": "^2.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/css-what": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz",
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">= 6"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/cssesc": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/cssesc/-/cssesc-3.0.0.tgz",
@@ -4947,6 +5018,73 @@
"integrity": "sha512-+HlytyjlPKnIG8XuRG8WvmBP8xs8P71y+SKKS6ZXWoEgLuePxtDoUEiH7WkdePWrQ5JBpE6aoVqfZfJUQkjXwA==",
"license": "MIT"
},
"node_modules/dom-serializer": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.2",
"entities": "^4.2.0"
},
"funding": {
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/dom-serializer/node_modules/entities": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/domelementtype": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause"
},
"node_modules/domhandler": {
"version": "5.0.3",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^2.3.0"
},
"engines": {
"node": ">= 4"
},
"funding": {
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/domutils": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^2.0.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3"
},
"funding": {
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
@@ -5058,6 +5196,31 @@
"node": ">= 0.8"
}
},
"node_modules/encoding-sniffer": {
"version": "0.2.1",
"resolved": "https://registry.npmjs.org/encoding-sniffer/-/encoding-sniffer-0.2.1.tgz",
"integrity": "sha512-5gvq20T6vfpekVtqrYQsSCFZ1wEg5+wW0/QaZMWkFr6BqD3NfKs0rLCx4rrVlSWJeZb5NBJgVLswK/w2MWU+Gw==",
"license": "MIT",
"dependencies": {
"iconv-lite": "^0.6.3",
"whatwg-encoding": "^3.1.1"
},
"funding": {
"url": "https://github.com/fb55/encoding-sniffer?sponsor=1"
}
},
"node_modules/encoding-sniffer/node_modules/iconv-lite": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz",
"integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==",
"license": "MIT",
"dependencies": {
"safer-buffer": ">= 2.1.2 < 3.0.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/end-of-stream": {
"version": "1.4.5",
"resolved": "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz",
@@ -6269,6 +6432,25 @@
],
"license": "MIT"
},
"node_modules/htmlparser2": {
"version": "10.1.0",
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz",
"integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==",
"funding": [
"https://github.com/fb55/htmlparser2?sponsor=1",
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.2",
"entities": "^7.0.1"
}
},
"node_modules/http-errors": {
"version": "2.0.1",
"resolved": "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz",
@@ -7473,7 +7655,6 @@
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
"dev": true,
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0"
@@ -7703,6 +7884,55 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/parse5": {
"version": "7.3.0",
"resolved": "https://registry.npmjs.org/parse5/-/parse5-7.3.0.tgz",
"integrity": "sha512-IInvU7fabl34qmi9gY8XOVxhYyMyuH2xUNpb2q8/Y+7552KlejkRvqvD19nMoUW/uQGGbqNpA6Tufu5FL5BZgw==",
"license": "MIT",
"dependencies": {
"entities": "^6.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5-htmlparser2-tree-adapter": {
"version": "7.1.0",
"resolved": "https://registry.npmjs.org/parse5-htmlparser2-tree-adapter/-/parse5-htmlparser2-tree-adapter-7.1.0.tgz",
"integrity": "sha512-ruw5xyKs6lrpo9x9rCZqZZnIUntICjQAd0Wsmp396Ul9lN/h+ifgVV1x1gZHi8euej6wTfpqX8j+BFQxF0NS/g==",
"license": "MIT",
"dependencies": {
"domhandler": "^5.0.3",
"parse5": "^7.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5-parser-stream": {
"version": "7.1.2",
"resolved": "https://registry.npmjs.org/parse5-parser-stream/-/parse5-parser-stream-7.1.2.tgz",
"integrity": "sha512-JyeQc9iwFLn5TbvvqACIF/VXG6abODeB3Fwmv/TGdLk2LfbWkaySGY72at4+Ty7EkPZj854u4CrICqNk2qIbow==",
"license": "MIT",
"dependencies": {
"parse5": "^7.0.0"
},
"funding": {
"url": "https://github.com/inikulin/parse5?sponsor=1"
}
},
"node_modules/parse5/node_modules/entities": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/entities/-/entities-6.0.1.tgz",
"integrity": "sha512-aN97NXWF6AWBTahfVOIrB/NShkzi5H7F9r1s9mD3cDj4Ko5f2qhhVoYMibXF7GlLveb/D2ioWay8lxI97Ven3g==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/parseurl": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
@@ -7868,6 +8098,18 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/playwright-core": {
"version": "1.61.1",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.61.1.tgz",
"integrity": "sha512-h7Qlt6m4REp25qvIdvbDtVmD4LqVXfpRxhORv9L0jzETM05p4fuPJ3dKyuSXQxDSbXnmS79HAgi9589lGSpLkg==",
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
}
},
"node_modules/pluralize": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/pluralize/-/pluralize-8.0.0.tgz",
@@ -9841,6 +10083,15 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/undici": {
"version": "7.28.0",
"resolved": "https://registry.npmjs.org/undici/-/undici-7.28.0.tgz",
"integrity": "sha512-cRZYrTDwWznlnRiPjggAGxZXanty6M8RV1ff8Wm4LWXBp7/IG8v5DnOm74DtUBp9OONpK75YlPnIjQqX0dBDtA==",
"license": "MIT",
"engines": {
"node": ">=20.18.1"
}
},
"node_modules/undici-types": {
"version": "7.24.6",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.24.6.tgz",
@@ -10140,6 +10391,40 @@
"vue": "^3.5.0"
}
},
"node_modules/whatwg-encoding": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/whatwg-encoding/-/whatwg-encoding-3.1.1.tgz",
"integrity": "sha512-6qN4hJdMwfYBtE3YBTTHhoeuUrDBPZmbQaxWAqSALV/MeEnR5z1xd8UKud2RAkFoPkmB+hli1TZSnyi84xz1vQ==",
"deprecated": "Use @exodus/bytes instead for a more spec-conformant and faster implementation",
"license": "MIT",
"dependencies": {
"iconv-lite": "0.6.3"
},
"engines": {
"node": ">=18"
}
},
"node_modules/whatwg-encoding/node_modules/iconv-lite": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.6.3.tgz",
"integrity": "sha512-4fCk79wshMdzMp2rH06qWrJE4iolqLhCUH+OiuIgU++RB0+94NlDL81atO7GX55uUKueo0txHNtvEyI6D7WdMw==",
"license": "MIT",
"dependencies": {
"safer-buffer": ">= 2.1.2 < 3.0.0"
},
"engines": {
"node": ">=0.10.0"
}
},
"node_modules/whatwg-mimetype": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/whatwg-mimetype/-/whatwg-mimetype-4.0.0.tgz",
"integrity": "sha512-QaKxh0eNIi2mE9p2vEdzfagOKHCcj1pJ56EEHGQOVxp8r9/iszLUUV7v89x9O1p/T+NlTM5W7jW6+cz4Fq1YVg==",
"license": "MIT",
"engines": {
"node": ">=18"
}
},
"node_modules/which": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/which/-/which-2.0.2.tgz",
+2
View File
@@ -81,9 +81,11 @@
"@vue/server-renderer": "^3.5.35",
"@vuepic/vue-datepicker": "^13.0.0",
"better-sqlite3": "^12.10.0",
"cheerio": "^1.2.0",
"edge.js": "^6.5.1",
"luxon": "^3.7.2",
"mysql2": "^3.22.4",
"playwright-core": "^1.61.1",
"reflect-metadata": "^0.2.2",
"uuid": "^14.0.1",
"vue": "^3.5.35",
+19
View File
@@ -0,0 +1,19 @@
/*
|--------------------------------------------------------------------------
| Headless browser lifecycle
|--------------------------------------------------------------------------
|
| The merchant price comparison feature (app/services/merchant_price_service.ts)
| launches a single shared headless Chromium instance on demand (see
| app/services/headless_browser_service.ts). This makes sure that process
| is closed when the app shuts down, instead of leaking a Chromium child
| process.
|
*/
import app from '@adonisjs/core/services/app'
app.terminating(async () => {
const { closeBrowser } = await import('#services/headless_browser_service')
await closeBrowser()
})
+3
View File
@@ -24,4 +24,7 @@ export default await Env.create(new URL('../', import.meta.url), {
// Session
SESSION_DRIVER: Env.schema.enum(['cookie', 'memory', 'database'] as const),
// Merchant price comparison (headless browser scraping)
CHROMIUM_EXECUTABLE_PATH: Env.schema.string.optional(),
})
+21 -3
View File
@@ -11,9 +11,11 @@ import { middleware } from '#start/kernel'
import { controllers } from '#generated/controllers'
import router from '@adonisjs/core/services/router'
router.get('/', async ({ response }) => {
return response.redirect('/auth/login')
}).as('home')
router
.get('/', async ({ response }) => {
return response.redirect('/auth/login')
})
.as('home')
router
.group(() => {
@@ -42,10 +44,16 @@ router
router.get('/:id/edit', [controllers.Computers, 'edit']).as('computers.edit')
router.put('/:id', [controllers.Computers, 'update']).as('computers.update')
router.delete('/:id', [controllers.Computers, 'destroy']).as('computers.destroy')
router.put('/:id/share', [controllers.ComputerShare, 'update']).as('computers.share.update')
router
.post('/:id/share/rotate', [controllers.ComputerShare, 'rotate'])
.as('computers.share.rotate')
})
.prefix('/computers')
.use(middleware.auth())
router.get('/share/:token', [controllers.PublicComputer, 'show']).as('share.show')
router
.group(() => {
router.get('/', [controllers.Components, 'index']).as('components.index')
@@ -58,3 +66,13 @@ router
})
.prefix('/components')
.use(middleware.auth())
router
.get('/scraped-parts', [controllers.ScrapedParts, 'index'])
.as('scraped_parts.index')
.use(middleware.auth())
router
.get('/scraped-parts/compare', [controllers.ScrapedParts, 'compare'])
.as('scraped_parts.compare')
.use(middleware.auth())
+94
View File
@@ -102,3 +102,97 @@ test.group('Components authorization (IDOR)', () => {
assert.isNotNull(stillExists)
})
})
test.group('Components purchase URL', () => {
test('a component can be created with a purchase site URL', async ({ client, assert }) => {
const user = await createUser('purchase-url-a@example.com')
const type = await createType('RAM-purchase-1')
const response = await client
.post('/components')
.loginAs(user)
.withCsrfToken()
.redirects(0)
.json({
name: 'New GPU',
price: 599.99,
typeId: type.id,
purchaseDate: DateTime.now().toFormat('yyyy-MM-dd'),
purchaseUrl: 'https://fr.pc-builder.io/gpu/example-p1',
quantity: 1,
})
response.assertStatus(302)
const component = await Component.query().where('userId', user.id).firstOrFail()
assert.equal(component.purchaseUrl, 'https://fr.pc-builder.io/gpu/example-p1')
})
test('a component can be created without a purchase site URL', async ({ client, assert }) => {
const user = await createUser('purchase-url-b@example.com')
const type = await createType('RAM-purchase-2')
const response = await client
.post('/components')
.loginAs(user)
.withCsrfToken()
.redirects(0)
.json({
name: 'Used RAM stick',
price: 20,
typeId: type.id,
purchaseDate: DateTime.now().toFormat('yyyy-MM-dd'),
quantity: 1,
})
response.assertStatus(302)
const component = await Component.query().where('userId', user.id).firstOrFail()
assert.isNull(component.purchaseUrl)
})
test('an invalid purchase URL is rejected', async ({ client, assert }) => {
const user = await createUser('purchase-url-c@example.com')
const type = await createType('RAM-purchase-3')
const response = await client
.post('/components')
.loginAs(user)
.withCsrfToken()
.redirects(0)
.json({
name: 'Suspicious component',
price: 20,
typeId: type.id,
purchaseDate: DateTime.now().toFormat('yyyy-MM-dd'),
purchaseUrl: 'not-a-url',
quantity: 1,
})
response.assertStatus(302)
assert.property(response.flashMessage('inputErrorsBag'), 'purchaseUrl')
const created = await Component.query().where('userId', user.id).first()
assert.isNull(created)
})
test('the purchase URL can be updated on an existing component', async ({ client, assert }) => {
const owner = await createUser('purchase-url-d@example.com')
const type = await createType('RAM-purchase-4')
const component = await createComponent(owner.id, type.id)
const response = await client
.put(`/components/${component.id}`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({
name: component.name,
price: component.price,
typeId: type.id,
purchaseDate: DateTime.now().toFormat('yyyy-MM-dd'),
purchaseUrl: 'https://fr.pc-builder.io/cpu/example-p2',
})
response.assertStatus(302)
await component.refresh()
assert.equal(component.purchaseUrl, 'https://fr.pc-builder.io/cpu/example-p2')
})
})
+170
View File
@@ -0,0 +1,170 @@
import { test } from '@japa/runner'
import User from '#models/user'
import Computer from '#models/computer'
import ComputerState from '#models/computer_state'
async function createUser(email: string) {
return User.create({
name: 'Test User',
email,
password: 'secret@123A',
})
}
async function createState(name = 'ON_GOING') {
return ComputerState.firstOrCreate({ name }, { name })
}
async function createComputer(userId: number, stateId: number) {
return Computer.create({
title: 'Shared PC',
sellPrice: 999,
gpuScore: 5,
cpuScore: 5,
globalScore: 5,
stateId,
userId,
})
}
test.group('Computer sharing', () => {
test('owner can enable sharing and the public page becomes reachable', async ({
client,
assert,
}) => {
const owner = await createUser('share-owner-a@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
const enableResponse = await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: true, showCost: false })
enableResponse.assertStatus(302)
await computer.refresh()
assert.isTrue(computer.shareEnabled)
assert.isString(computer.shareToken)
const publicResponse = await client
.get(`/share/${computer.shareToken}`)
.header('X-Inertia', 'true')
.header('X-Inertia-Version', '1')
.redirects(0)
publicResponse.assertStatus(200)
publicResponse.assertBodyContains({ props: { computer: { title: 'Shared PC' } } })
})
test('sell price and cost are hidden unless explicitly enabled', async ({ client }) => {
const owner = await createUser('share-owner-b@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: false, showCost: false })
await computer.refresh()
const publicResponse = await client
.get(`/share/${computer.shareToken}`)
.header('X-Inertia', 'true')
.header('X-Inertia-Version', '1')
.redirects(0)
publicResponse.assertStatus(200)
publicResponse.assertBodyContains({ props: { computer: { sellPrice: null, cost: null } } })
})
test('disabling sharing makes the public link unreachable', async ({ client }) => {
const owner = await createUser('share-owner-c@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: false, showCost: false })
await computer.refresh()
const token = computer.shareToken
await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: false, showSellPrice: false, showCost: false })
const publicResponse = await client.get(`/share/${token}`).redirects(0)
publicResponse.assertStatus(404)
})
test('a user cannot enable sharing on another user computer', async ({ client }) => {
const owner = await createUser('share-owner-d@example.com')
const attacker = await createUser('share-attacker-d@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
const response = await client
.put(`/computers/${computer.id}/share`)
.loginAs(attacker)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: true, showCost: true })
response.assertStatus(404)
})
test('rotating the share token invalidates the previous link', async ({ client, assert }) => {
const owner = await createUser('share-owner-e@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: false, showCost: false })
await computer.refresh()
const oldToken = computer.shareToken
await client
.post(`/computers/${computer.id}/share/rotate`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
await computer.refresh()
assert.notEqual(computer.shareToken, oldToken)
const oldLinkResponse = await client.get(`/share/${oldToken}`).redirects(0)
oldLinkResponse.assertStatus(404)
const newLinkResponse = await client.get(`/share/${computer.shareToken}`).redirects(0)
newLinkResponse.assertStatus(200)
})
test('an unauthenticated visitor can view a shared computer', async ({ client }) => {
const owner = await createUser('share-owner-f@example.com')
const state = await createState()
const computer = await createComputer(owner.id, state.id)
await client
.put(`/computers/${computer.id}/share`)
.loginAs(owner)
.withCsrfToken()
.redirects(0)
.json({ enabled: true, showSellPrice: true, showCost: true })
await computer.refresh()
// No .loginAs() here: simulates a fully anonymous visitor.
const response = await client.get(`/share/${computer.shareToken}`).redirects(0)
response.assertStatus(200)
})
})
@@ -0,0 +1,46 @@
import { test } from '@japa/runner'
import { extractModelToken, labelMatchesModelToken } from '#services/merchant_price_service'
test.group('extractModelToken', () => {
test('extracts a bare model number', ({ assert }) => {
assert.equal(extractModelToken('Samsung 970 Evo Plus 1 TB Solid State Drive'), '970')
})
test('extracts an alphanumeric model number', ({ assert }) => {
assert.equal(extractModelToken('AMD Ryzen 3 3200G 3.6 GHz Quad-Core Processor'), '3200G')
})
test('appends a known suffix immediately following the model number', ({ assert }) => {
assert.equal(extractModelToken('NVIDIA GeForce RTX 5060 Ti 16 GB Video Card'), '5060 Ti')
assert.equal(extractModelToken('NVIDIA GeForce RTX 4070 Super 12 GB Video Card'), '4070 Super')
})
test('does not append a suffix when the model number has none', ({ assert }) => {
assert.equal(extractModelToken('NVIDIA GeForce RTX 5060 8 GB Video Card'), '5060')
})
test('returns null when no model-like token is found', ({ assert }) => {
assert.isNull(extractModelToken('Some Generic Product Without Any Model Number'))
})
})
test.group('labelMatchesModelToken', () => {
test('a bare model token matches a label with no suffix', ({ assert }) => {
assert.isTrue(labelMatchesModelToken('MSI VENTUS 2X OC NVIDIA GeForce RTX 5060 8G', '5060'))
})
test('a bare model token does NOT match a label carrying a known suffix', ({ assert }) => {
// Regression: RTX 5060 vs RTX 5060 Ti must never be conflated - they
// are different, differently-priced cards.
assert.isFalse(labelMatchesModelToken('PNY NVIDIA GeForce RTX 5060 Ti 8G OC', '5060'))
})
test('a suffixed model token only matches labels with that exact suffix', ({ assert }) => {
assert.isTrue(labelMatchesModelToken('PNY NVIDIA GeForce RTX 5060 Ti 8G OC', '5060 Ti'))
assert.isFalse(labelMatchesModelToken('MSI VENTUS 2X OC NVIDIA GeForce RTX 5060 8G', '5060 Ti'))
})
test('returns false when the token is not present at all', ({ assert }) => {
assert.isFalse(labelMatchesModelToken('AMD Radeon RX 7800 XT', '5060'))
})
})
+301
View File
@@ -0,0 +1,301 @@
import { test } from '@japa/runner'
import { DateTime } from 'luxon'
import User from '#models/user'
import ComponentType from '#models/component_type'
import ScrapedPart from '#models/scraped_part'
import {
isCategoryStale,
ensureFreshCategory,
updatePriceIfCheaper,
clearPriceIfNoOffers,
shouldKeepExistingPrice,
CACHE_TTL_HOURS,
} from '#services/part_scraper_service'
async function createUser(email: string) {
return User.create({
name: 'Test User',
email,
password: 'secret@123A',
})
}
async function createType(name: string, unique = true) {
return ComponentType.firstOrCreate({ name }, { name, unique })
}
async function createScrapedPart(
typeId: number,
name: string,
sourceRef: string,
price: number | null = 199.99
) {
return ScrapedPart.create({
typeId,
name,
price,
imageUrl: 'https://fr.pc-builder.io/images/cpu/example.jpg',
source: 'pc-builder.io',
sourceUrl: `https://fr.pc-builder.io/cpu/${sourceRef}`,
sourceRef,
scrapedAt: DateTime.now(),
})
}
test.group('Scraped parts search', () => {
test('an authenticated user can search scraped parts by name', async ({ client, assert }) => {
const user = await createUser('scraped-search-a@example.com')
const type = await createType('Processor-scraped-1')
await createScrapedPart(type.id, 'ZZZ-Test-Ryzen-Unique 5 7600X', 'amd-ryzen-5-7600x-p1311')
await createScrapedPart(type.id, 'ZZZ-Test-Intel-Unique i5-13600K', 'intel-i5-13600k-p9999')
const response = await client.get('/scraped-parts?q=ZZZ-Test-Ryzen-Unique').loginAs(user)
response.assertStatus(200)
const body = response.body()
assert.lengthOf(body, 1)
assert.equal(body[0].name, 'ZZZ-Test-Ryzen-Unique 5 7600X')
assert.equal(body[0].type.id, type.id)
})
test('search can be filtered by typeId', async ({ client, assert }) => {
const user = await createUser('scraped-search-b@example.com')
const cpuType = await createType('Processor-scraped-2')
const gpuType = await createType('Graphic Card-scraped-2')
await createScrapedPart(cpuType.id, 'ZZZ-Test-AMD Ryzen 7 7800X3D', 'amd-ryzen-7-7800x3d-p1')
await createScrapedPart(gpuType.id, 'ZZZ-Test-AMD Radeon RX 7800', 'amd-radeon-rx-7800-p2')
const response = await client
.get(`/scraped-parts?q=ZZZ-Test-AMD&typeId=${gpuType.id}`)
.loginAs(user)
response.assertStatus(200)
const body = response.body()
assert.lengthOf(body, 1)
assert.equal(body[0].name, 'ZZZ-Test-AMD Radeon RX 7800')
})
test('an unauthenticated request is redirected to login', async ({ client }) => {
const response = await client.get('/scraped-parts?q=ryzen')
response.assertRedirectsTo('/auth/login')
})
test('parts without a known price are kept and sorted last', async ({ client, assert }) => {
const user = await createUser('scraped-search-c@example.com')
const type = await createType('Graphic Card-scraped-3')
await createScrapedPart(type.id, 'ZZZ-Test-RTX Cheap', 'rtx-cheap-p1', 300)
await createScrapedPart(type.id, 'ZZZ-Test-RTX Expensive', 'rtx-expensive-p2', 900)
await createScrapedPart(type.id, 'ZZZ-Test-RTX NoPrice', 'rtx-noprice-p3', null)
const response = await client.get('/scraped-parts?q=ZZZ-Test-RTX').loginAs(user)
response.assertStatus(200)
const body = response.body()
assert.lengthOf(body, 3)
assert.equal(body[0].name, 'ZZZ-Test-RTX Cheap')
assert.equal(body[1].name, 'ZZZ-Test-RTX Expensive')
assert.equal(body[2].name, 'ZZZ-Test-RTX NoPrice')
assert.isNull(body[2].price)
})
})
test.group('Scraped parts 24h cache freshness', () => {
test('a category with no cached parts is considered stale', async ({ assert }) => {
const type = await createType('Processor-freshness-1')
assert.isTrue(await isCategoryStale(type))
})
test('a category scraped within the last 24h is not stale', async ({ assert }) => {
const type = await createType('Processor-freshness-2')
await ScrapedPart.create({
typeId: type.id,
name: 'ZZZ-Test-Fresh Part',
price: 100,
imageUrl: null,
source: 'pc-builder.io',
sourceUrl: 'https://fr.pc-builder.io/cpu/fresh-part',
sourceRef: 'fresh-part-ref',
scrapedAt: DateTime.now().minus({ hours: 1 }),
})
assert.isFalse(await isCategoryStale(type))
})
test('a category scraped more than 24h ago is stale', async ({ assert }) => {
const type = await createType('Processor-freshness-3')
await ScrapedPart.create({
typeId: type.id,
name: 'ZZZ-Test-Stale Part',
price: 100,
imageUrl: null,
source: 'pc-builder.io',
sourceUrl: 'https://fr.pc-builder.io/cpu/stale-part',
sourceRef: 'stale-part-ref',
scrapedAt: DateTime.now().minus({ hours: CACHE_TTL_HOURS + 1 }),
})
assert.isTrue(await isCategoryStale(type))
})
test('ensureFreshCategory does not scrape unmapped types (no network call)', async ({
assert,
}) => {
// "Other" is a real, unmapped ComponentType (see CATEGORY_SLUGS): this
// must resolve without making any outbound HTTP request.
const type = await createType('Other-freshness-unmapped', false)
await assert.doesNotReject(() => ensureFreshCategory(type))
})
})
test.group('Scraped parts merchant price comparison', () => {
test('an unauthenticated request is redirected to login', async ({ client }) => {
const response = await client.get('/scraped-parts/compare?id=1')
response.assertRedirectsTo('/auth/login')
})
test('comparing an unknown scraped part returns 404', async ({ client }) => {
const user = await createUser('compare-unknown@example.com')
const response = await client.get('/scraped-parts/compare?id=999999999').loginAs(user)
response.assertStatus(404)
})
test('an invalid id is rejected by validation', async ({ client }) => {
const user = await createUser('compare-invalid@example.com')
const response = await client
.get('/scraped-parts/compare?id=not-a-number')
.header('Accept', 'application/json')
.loginAs(user)
response.assertStatus(422)
})
})
test.group('Best-known-price persistence', () => {
test('a cheaper offer replaces the current price', async ({ assert }) => {
const type = await createType('Graphic Card-price-1')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Price-A', 'price-a-p1', 500)
await updatePriceIfCheaper(part, { merchant: 'LDLC', price: 350, url: 'https://ldlc.com/x' })
await part.refresh()
assert.equal(part.price, 350)
assert.equal(part.priceSource, 'LDLC')
assert.equal(part.sourceUrl, 'https://ldlc.com/x')
assert.isNotNull(part.priceCheckedAt)
})
test('a more expensive offer does not replace the current price', async ({ assert }) => {
const type = await createType('Graphic Card-price-2')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Price-B', 'price-b-p1', 300)
await updatePriceIfCheaper(part, { merchant: 'LDLC', price: 450, url: 'https://ldlc.com/y' })
await part.refresh()
assert.equal(part.price, 300)
assert.isNull(part.priceSource)
})
test('an offer is accepted when no price was known yet', async ({ assert }) => {
const type = await createType('Graphic Card-price-3')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Price-C', 'price-c-p1', null)
await updatePriceIfCheaper(part, {
merchant: 'Grosbill',
price: 599.99,
url: 'https://grosbill.com/z',
})
await part.refresh()
assert.equal(part.price, 599.99)
assert.equal(part.priceSource, 'Grosbill')
})
})
test.group('clearPriceIfNoOffers', () => {
test('clears an unverified pc-builder.io catalog price when no merchant confirms it', async ({
assert,
}) => {
// Regression: memory sticks kept showing pc-builder.io catalog prices
// (e.g. €35.99) even though comparing prices found no matching offer
// on any merchant ("No live price found, using catalog price €35.99")
// — an unconfirmed number should not be displayed as if it were still
// valid.
const type = await createType('Memory-clear-1')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Clear-A', 'clear-a-p1', 35.99)
await clearPriceIfNoOffers(part)
await part.refresh()
assert.isNull(part.price)
assert.isNull(part.priceSource)
})
test('also clears a price previously verified by a merchant if no longer confirmed', async ({
assert,
}) => {
const type = await createType('Memory-clear-2')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Clear-B', 'clear-b-p1', 50)
part.priceSource = 'LDLC'
await part.save()
await clearPriceIfNoOffers(part)
await part.refresh()
assert.isNull(part.price)
assert.isNull(part.priceSource)
})
test('is a no-op when the part already has no price', async ({ assert }) => {
const type = await createType('Memory-clear-3')
const part = await createScrapedPart(type.id, 'ZZZ-Test-Clear-C', 'clear-c-p1', null)
const originalCheckedAt = part.priceCheckedAt
await clearPriceIfNoOffers(part)
await part.refresh()
assert.isNull(part.price)
assert.equal(part.priceCheckedAt, originalCheckedAt)
})
})
test.group('Catalog refresh price protection (shouldKeepExistingPrice)', () => {
test('an unverified pc-builder.io price is NOT protected and gets overwritten', ({ assert }) => {
// Regression test: a GPU had price=431.17 from a previous pc-builder.io
// catalog scrape (priceSource: null, never independently verified).
// pc-builder.io later stopped showing a fixed price for it (only a
// generic Amazon search link). The stale 431.17 must NOT be kept.
const existing = { price: 431.17, priceSource: null }
assert.isFalse(shouldKeepExistingPrice(existing, null))
assert.isFalse(shouldKeepExistingPrice(existing, 999))
})
test('a verified merchant price IS protected from a worse/missing catalog price', ({
assert,
}) => {
const existing = { price: 350, priceSource: 'LDLC' }
assert.isTrue(shouldKeepExistingPrice(existing, null))
assert.isTrue(shouldKeepExistingPrice(existing, 500))
})
test('a verified merchant price is replaced if the catalog price is actually cheaper', ({
assert,
}) => {
const existing = { price: 350, priceSource: 'LDLC' }
assert.isFalse(shouldKeepExistingPrice(existing, 200))
})
test('no existing part means nothing to keep', ({ assert }) => {
assert.isFalse(shouldKeepExistingPrice(null, 100))
assert.isFalse(shouldKeepExistingPrice(null, null))
})
})