Kevin L'Hereec
5854ff8ed1
Idealo and PCPartPicker were evaluated and ruled out: - idealo.fr search pages return 503 (Cloudflare) - the internal Idealo GraphQL endpoint used by a public scraper (app.idealo.de/app-backend/api) is protected by Akamai Bot Manager and silently returns empty data even with mobile app headers - PCPartPicker's product list/search relies on a session-bound CSRF token refreshed per page load, against a scraping-hostile ToS pc-builder.io category pages are server-rendered, include name/price/image for ~100 products per category, and have no bot protection, so the feature is built on top of that source only. - New scraped_parts cache table (type_id, name, price, image_url, source, source_url, source_ref unique per source) populated by a batch job rather than scraped live per search, since only ~100 first results per category are available without reverse-engineering pc-builder.io's DataTables/CSRF pagination flow - part_scraper_service.ts: maps our ComponentType names to pc-builder.io category slugs (SSD -> storage, since pc-builder.io has no separate HDD category) and parses each category page with cheerio, skipping rows without a resolved price (generic 'search on Amazon' links) - node ace scrape:parts: refreshes the cache for every mapped category (upsert on source+source_ref), runnable manually or later via cron - GET /scraped-parts (authenticated): search the cache by name (LOWER() LIKE, avoids a MySQL utf8_bin/utf8mb4 collation mismatch from Lucid's whereLike) and optional typeId filter - NewPartSearch.vue: debounced search widget wired into components/create.vue; clicking 'Import' pre-fills the existing component creation form (name, price, type, purchase date = today) instead of creating the Component directly, so the user still reviews and saves it like any manually entered part - 3 functional tests covering search by name, filter by typeId, and unauthenticated access being redirected to login - Initial scrape run: 989 parts across 10 categories
74 lines
2.7 KiB
TypeScript
74 lines
2.7 KiB
TypeScript
/*
|
|
|--------------------------------------------------------------------------
|
|
| Routes file
|
|
|--------------------------------------------------------------------------
|
|
|
|
|
| The routes file is used for defining the HTTP routes.
|
|
|
|
|
*/
|
|
|
|
import { middleware } from '#start/kernel'
|
|
import { controllers } from '#generated/controllers'
|
|
import router from '@adonisjs/core/services/router'
|
|
|
|
router
|
|
.get('/', async ({ response }) => {
|
|
return response.redirect('/auth/login')
|
|
})
|
|
.as('home')
|
|
|
|
router
|
|
.group(() => {
|
|
router.get('signup', [controllers.NewAccount, 'create']).as('new_account.create')
|
|
router.post('signup', [controllers.NewAccount, 'store']).as('new_account.store')
|
|
|
|
router.get('login', [controllers.Session, 'create']).as('session.create')
|
|
router.post('login', [controllers.Session, 'store']).as('session.store')
|
|
})
|
|
.prefix('/auth')
|
|
.use(middleware.guest())
|
|
|
|
router
|
|
.group(() => {
|
|
router.post('logout', [controllers.Session, 'destroy'])
|
|
})
|
|
.prefix('/auth')
|
|
.use(middleware.auth())
|
|
|
|
router
|
|
.group(() => {
|
|
router.get('/', [controllers.Computers, 'index']).as('computers.index')
|
|
router.get('/create', [controllers.Computers, 'create']).as('computers.create')
|
|
router.post('/', [controllers.Computers, 'store']).as('computers.store')
|
|
router.get('/:id', [controllers.Computers, 'show']).as('computers.show')
|
|
router.get('/:id/edit', [controllers.Computers, 'edit']).as('computers.edit')
|
|
router.put('/:id', [controllers.Computers, 'update']).as('computers.update')
|
|
router.delete('/:id', [controllers.Computers, 'destroy']).as('computers.destroy')
|
|
router.put('/:id/share', [controllers.ComputerShare, 'update']).as('computers.share.update')
|
|
router
|
|
.post('/:id/share/rotate', [controllers.ComputerShare, 'rotate'])
|
|
.as('computers.share.rotate')
|
|
})
|
|
.prefix('/computers')
|
|
.use(middleware.auth())
|
|
|
|
router.get('/share/:token', [controllers.PublicComputer, 'show']).as('share.show')
|
|
|
|
router
|
|
.group(() => {
|
|
router.get('/', [controllers.Components, 'index']).as('components.index')
|
|
router.get('/create', [controllers.Components, 'create']).as('components.create')
|
|
router.post('/', [controllers.Components, 'store']).as('components.store')
|
|
router.get('/:id', [controllers.Components, 'show']).as('components.show')
|
|
router.get('/:id/edit', [controllers.Components, 'edit']).as('components.edit')
|
|
router.put('/:id', [controllers.Components, 'update']).as('components.update')
|
|
router.delete('/:id', [controllers.Components, 'destroy']).as('components.destroy')
|
|
})
|
|
.prefix('/components')
|
|
.use(middleware.auth())
|
|
|
|
router
|
|
.get('/scraped-parts', [controllers.ScrapedParts, 'index'])
|
|
.as('scraped_parts.index')
|
|
.use(middleware.auth())
|