All changes are on local branch feature/HOI-4097-watchlist-batch-display (commits 01a5b1a9cd, 553f72cbd3, e9aa11cc8b, e061846ca8, 90887b95e1, e9f1ed4501, 60593be6a6, 7ec063a738), not pushed. Numbers are medians of 3 interleaved rounds per arm, 0 errors, unless noted. Expand a change for the problem, the fix, the code and its proof.
Accepted changes
1. Watch-list: one batch call per page — watch-list
- Problem
- Core called
GET /contents?contentId=once per watch-list item. Each call loaded the full title, and for a series that meant every episode, the cast and recommendations, just to show a card. - Fix
- New
POST /contents/watch-list-displaytakes the page's{contentId, contentType}items, groups them by type and reads each type once through the same cached batch services the single-title route uses (≤25 ids per call). Series are read without the episodes/people/recommendations joins. Core calls it once per page (chunks of 100). - Files
apps/coreapis/.../user-profile.service.ts,apps/contents/.../content.service.ts,contents.controller.ts,get-watch-list-content-display.request.ts, content API client
+ watchListPagedResponse.watchList = await this.enrichWatchList(watchListResponses ?? [], {
+ language: query.language,
+ platform: query.platform,
+ appVersion: query.appVersion,
+ });
+ return watchListPagedResponse;
+ }
+
+ /**
+ * One content call per watch-list page instead of one per item. Keeps the previous outcome per
+ * item: content without a published record is dropped (was the per-item 404), and if the batch
+ * call itself fails every item is returned unenriched (was the per-item non-404 error path).
+ */
+ private async enrichWatchList(
+ watchList: UserWatchListResponse[],
+ context: { language?: string; platform?: string; appVersion?: string },
+ ): Promise<UserWatchListResponse[]> {
+ if (!watchList.length) return watchList;
+ let displays: WatchListContentDisplay[];
+ try {
+ // Content accepts up to 100 items per call; `items` per page is client-controlled.
+ const chunks: UserWatchListResponse[][] = [];
+ for (let i = 0; i < watchList.length; i += 100) chunks.push(watchList.slice(i, i + 100));
+ const responses = await Promise.all(
+ chunks.map((chunk) =>
+ this.contentServiceApiClient.getWatchListContentDisplay(
+ chunk.map(({ contentId, contentType }) => ({ contentId, contentType })),
+ context,
+ ),
+ ),
+ );
+ displays = responses.flatMap((response) => response?.data ?? []);
+ } catch (error) {
+ this.logger.error('Error while enriching watch-list content', { err: error, items: watchList.length });
… (81 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| CPU per page (prod-shaped mix, Node 26) | 36.5 ms | 3.69 ms | −90% |
| Pages/s per pod | ≈ 43 | 343 | ≈ 8× |
- Output check
- 0 field differences on 72 cards (24 titles × 3 platform/language contexts). Unknown and unpublished ids are dropped by both.
2. Watch-list: card-sized cache for movies — watch-list
- Problem
- A watch-list card needs ~8 fields, but each movie card decoded the full cached response (~10 KB zstd: renditions, people, recommendations). Decoding was 22.5% of the main thread.
- Fix
- Cache a card-sized projection as field
videocard:<lang>inside the movie's existing Valkey hash. Unpublish deletes the whole hash, and publish re-derives the card from the refreshed full response, so it cannot go stale. - Files
content-movies.service.ts,cache-keys.constants.ts
+ * Card-sized reads for watch-list display. A full cached video response is ~10KB zstd (renditions,
+ * people, recommendations) and decoding it is most of the batch route's CPU; the card is <1KB.
+ * Cards live in the video's own hash ('videocard:<lang>'), so unpublish deletes them with the rest
+ * and refreshCache re-derives them. A miss is built from getVideosByContentId (its cache first).
+ */
+ public async getVideoCardsByContentId(ids: string[], language: Language): Promise<WatchListContentDisplay[]> {
+ if (!ids?.length) return [];
+ const cacheSuffix = `${VIDEO_CARD_GROUPED_CACHE_TYPE_PREFIX}${language}`;
+ const cached = await this.cacheService.getorLoadManyExternalSWR<CachedVideoCard>(
+ ids.map((id) => ({ id, type: cacheSuffix })),
+ async (missed: CacheKeyInterface[]) => {
+ const missedIds = missed.map((key) => key.id);
+ // Platform only filters video renditions, which cards don't carry.
+ const videos = (await this.getVideosByContentId(missedIds, language, Platform.WEB, { limit: Infinity })) ?? [];
+ const loaded = videos.map((video) => ({ id: video.contentId, type: cacheSuffix, value: toVideoCard(video) }));
+ return withNotFoundMarkers(missed, loaded);
+ },
+ );
+ return withoutNotFoundMarkers(cached.map((key) => key.value)).filter(Boolean);
+ }
+
public filterDRMRenditionsTemporarily(
@@ -287,3 +327,3 @@ export class ContentMoviesService implements ContentMoviesServiceInterface {
public async getGeoRestrictionsInfoByContentId(contentId: string): Promise<GeoRestriction> {
- const content = await this.videosRepository.findByContentId(contentId);
+ const content = await this.videosRepository.findByContentId(contentId, ['geoRestriction']);
return content?.geoRestriction;
@@ -346,2 +386,3 @@ export class ContentMoviesService implements ContentMoviesServiceInterface {
const responseFields = existingFields.filter((field) => field.startsWith('video:'));
+ const cardFields = existingFields.filter((field) => field.startsWith(VIDEO_CARD_GROUPED_CACHE_TYPE_PREFIX));
… (15 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| CPU per page (prod mix) | 3.58 ms | 2.52 ms | −30% |
| CPU per page (movie-only pages) | 4.55 ms | 2.11 ms | −54% |
| p99 | 95 ms | 64 ms |
- Output check
- 0 field differences on 219 movie cards × 3 contexts.
3. Internal calls ask for uncompressed responses — all routes called by core/analytics
- Problem
- axios asks for
br/gzipby default, so content brotli-compressed every response to core and analytics, and they decompressed it again. Inside the cluster, bandwidth is not the constraint; CPU is. - Fix
- The shared HTTP client sends
Accept-Encoding: identityon internal calls (not on calls marked external, and never over a caller's own header). Content already honours it. - Files
libs/hoichoi-common/.../hoichoi-http.service.ts
@@ -348,2 +348,8 @@ export class HoichoiHttpService implements HoichoiHttpServiceInterface {
}
+ // Service-to-service calls stay inside the cluster, where bandwidth is not the constraint but CPU
+ // is: axios asks for br/gzip by default, so the callee compresses and we decompress every body.
+ // Measured on content (HOI-4097): compression was 23-33% of its CPU per request.
+ if (!externalCall && !this.hasHeader(headers, 'accept-encoding')) {
+ headers['Accept-Encoding'] = 'identity';
+ }
const validateStatus =| Measure | Before | After | Change |
|---|---|---|---|
| Single title CPU/request | 3.35 ms | 2.15 ms | −36% |
| Watch-history display | 6.74 ms | 2.84 ms | −58% |
| Gates | 1.65 ms | 1.17 ms | −29% |
| Watch-list | 1.88 ms | 1.43 ms | −24% |
- Output check
- Body identical; response size goes from ~8 KB compressed to ~85 KB raw on the in-cluster network. Applies to 79% of content's requests.
4. Entitlement: one detector, one detection per request — entitlement
- Problem
detectDevicebuilt a newDeviceDetector(every parser and index) on every call, and the controller's Roku check ran a full detection that the service then repeated. Detection was ~46% of entitlement's main-thread CPU.- Fix
- One detector per process (it holds only fixed options and static tables;
detect()is synchronous). The controller keeps its detection result and passes it to the service. - Files
device-detector.helper.ts,entitlement.service.controller.ts,entitlement.service.ts
@@ -9,10 +9,14 @@ import { Platform } from '../enums/app-cms/pages.enum';
+// One detector per process: it holds only these fixed options and the library's static regex tables,
+// and detect() is synchronous, so sharing it is equivalent to building one per call (which re-instantiated
+// every parser on each request).
+const detector = new DeviceDetector({
+ clientIndexes: true,
+ deviceIndexes: true,
+ deviceAliasCode: true,
+ // deviceInfo: true, # Commenting this out, causing server issue @Krishan
+});
+
export class DeviceDetectorHelper implements DeviceDetectorHelperInterface {
public detectDevice(userAgent?: string): UserDeviceInfoDto {
- const detector = new DeviceDetector({
- clientIndexes: true,
- deviceIndexes: true,
- deviceAliasCode: true,
- // deviceInfo: true, # Commenting this out, causing server issue @Krishan
- });
// detect(undefined) throws inside the library; requests without a User-Agent header must not 500
@@ -93,6 +97,5 @@ export class DeviceDetectorHelper implements DeviceDetectorHelperInterface {
- public isRoku(userAgent: string): boolean {
+ public isRokuDevice(device: UserDeviceInfoDto): boolean {
try {
- const deviceMetaData: UserDeviceInfoDto = this.detectDevice(userAgent);
- const deviceInfo: DetectResult = JSON.parse(deviceMetaData.deviceMetaData);
+ const deviceInfo: DetectResult = JSON.parse(device.deviceMetaData);
return deviceInfo.device.brand.toLowerCase() === 'roku';| Measure | Before | After | Change |
|---|---|---|---|
| Entitlement CPU/request | 9.08 ms | 5.23 ms | −42% |
| Requests/s per pod | 122 | 219 | |
| p99 | 270 ms | 161 ms |
- Output check
- Same status and body on all 342 distinct scenario requests (199 × 200, 143 × 401). Detection output identical on 41,131 fixture user agents.
5. node-device-detector: compile regexes once, skip impossible brands — entitlement (and core login)
- Problem
- The library builds
new RegExp(...)from static patterns on every call, and for generic UAs (Android 30; AndroidTVwith no model) it tries every one of ~2,000 brand regexes. - Fix
- A
patch-packagepatch: memoise compiled regexes (non-global, so nolastIndexstate), and test chunks of 64 brands with one combined regex first, scanning a chunk only if it can match. First match and result are unchanged. - Files
patches/node-device-detector+2.2.7.patch
@@ -116,13 +139,21 @@ class DeviceParserAbstract extends ParserAbstract {
}
if (!output.length) {
- for (let cursor in this.collection) {
- const result = this.#parseForBrand(cursor, userAgent);
- if (result === null) {
- continue;
+ // HOI-4097 patch: full scan tries every brand regex in collection order. Test chunks of brands with
+ // one combined regex first and skip chunks with no possible match; inside a matching chunk the
+ // original per-brand order runs, so the first match (and result) is unchanged.
+ for (const chunk of this.#brandChunks()) {
+ if (chunk.re !== null && !chunk.re.test(userAgent)) continue;
+ let stop = false;
+ for (const cursor of chunk.cursors) {
+ const result = this.#parseForBrand(cursor, userAgent);
+ if (result === null) {
+ continue;
+ }
+ output.push(result);
+ if (canBreak) { stop = true; break; }
}
- output.push(result);
…
* @param userAgent
* @returns {RegExpExecArray}
*/
+// HOI-4097 patch: patterns come from the library's static rule set, so compile each once.
+// Non-global ('i' only) regexes keep no lastIndex state, so a shared instance is equivalent.
+const MATCH_UA_CACHE = new Map();
+const BASE_RE_CACHE = new Map();
+const NORMALIZE_CACHE = new Map();
function matchUserAgent(str, userAgent) {
- let regex = '(?:^|[^A-Z_-])(?:' + normalizeRegExp(str) + ')';
- let match = new RegExp(regex, 'i');
+ let match = MATCH_UA_CACHE.get(str);
+ if (match === undefined) {
+ match = new RegExp('(?:^|[^A-Z_-])(?:' + normalizeRegExp(str) + ')', 'i');
+ MATCH_UA_CACHE.set(str, match);| Measure | Before | After | Change |
|---|---|---|---|
| detect() on prod user agents | 7.2 ms | 1.9 ms | −74% |
| detect() on 40,186 fixture UAs | 2.48 ms | 1.80 ms | −27% |
- Output check
- 0 differences on 40,186 upstream fixture user agents. Needs
patch-package+postinstallin the repo (decision pending).
6. Anonymous token key built once — entitlement + anon session middleware
- Problem
- Given a string secret, jsonwebtoken calls
createPublicKey(secret)on every verify, which throws for an HMAC secret, then falls back tocreateSecretKey. Two callers per request. - Fix
- Build the key once with the same resolution order; one entry per secret value, re-derived if the env value changes.
- Files
anon-token.util.ts,entitlement-auth.guard.ts
@@ -1,3 +1,27 @@
+import { createPublicKey, createSecretKey, KeyObject } from 'node:crypto';
import { verify as jwtVerify } from 'jsonwebtoken';
+let cachedKey: { secret: string; key: KeyObject } | undefined;
+
+/**
+ * CMS_JWT_TOKEN_SECRET as the KeyObject jsonwebtoken.verify would build from the string. Given a string,
+ * jsonwebtoken first tries createPublicKey (which throws for an HMAC secret) and then createSecretKey, on
+ * every verify. Same resolution order, done once per distinct secret value (one entry, re-derived if the
+ * env value changes). Undefined when the secret is unset, so callers keep their "no secret" behaviour.
+ */
+export function anonTokenVerifyKey(): KeyObject | undefined {
+ const secret = process.env.CMS_JWT_TOKEN_SECRET;
+ if (!secret) return undefined;
+ if (cachedKey?.secret !== secret) {
+ let key: KeyObject;
+ try {
+ key = createPublicKey(secret);
+ } catch {
+ key = createSecretKey(Buffer.from(secret));
+ }
+ cachedKey = { secret, key };
+ }
+ return cachedKey.key;
+}
+
export function isValidAnonId(value: unknown): value is string {
@@ -29,3 +53,3 @@ export function extractAnonIdFromAnonymousToken(
if (typeof token !== 'string' || token.length === 0) return undefined;
… (3 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Entitlement CPU/request | 5.58 ms | 5.01 ms | −10% |
- Output check
- Valid, tampered, garbage and unsigned tokens give identical status and body.
7. Geo check reads one field — single title
- Problem
- Every single-title request ran
findOneBy({contentId}), fetching the whole document from Mongo just to readgeoRestriction. - Fix
- Project only
geoRestriction(same query, same freshness, no cache added). - Files
- 6 content repositories + their services
@@ -40,2 +40,12 @@ export class ContentVideosRepository extends BaseRepository<ContentVideos> imple
+ /** contentType only, for type resolution (avoids decoding the whole video document). */
+ async getVideoContentType(contentId: string): Promise<string | undefined> {
+ const [doc] = await this.videosRepository.manager
+ .createEntityCursor<ContentVideos>(ContentVideos, { contentId })
+ .project({ contentType: 1 })
+ .limit(1)
+ .toArray();
+ return doc?.contentType;
+ }
+
async getVideoContent(contentId: string): Promise<ContentVideos> {
@@ -175,4 +185,5 @@ export class ContentVideosRepository extends BaseRepository<ContentVideos> imple
… (7 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Single title CPU/request | 4.67 ms | 4.10 ms | −12% |
| Requests/s | 329 | 384 |
- Output check
- Geo enforcement unchanged: a BD-excluded movie and series still return 403 for BD and 200 for IN.
8. contentData reads only what it returns — contentData
- Problem
- To return a type and season/episode ids, it loaded a full movie, then on a miss a full series (~42 KB), and ran
plainToInstanceover it. - Fix
- Two projected reads: the movie's
contentType(~ID only), then the series' type + season/episode ids (~1 KB). Same movie-first order and 404 semantics. - Files
content.service.ts,content-videos.repository.ts,content-series.repository.ts
public async getContentData(contentId: string): Promise<ContentInformationDto> {
- let movieContentData: ContentVideoDto;
- try {
- movieContentData = await this.contentMoviesService.getMovieContent(contentId);
- if (movieContentData) {
- return { contentType: movieContentData.contentType };
- }
- } catch (error) {
- this.logger.info(`No movie content data for contentId: ${contentId}`);
- }
- let seriesContentData: ContentSeriesDto;
- try {
- seriesContentData = await this.contentSeriesService.getSeriesContent(contentId);
- } catch (error) {
- this.logger.info(`No series content data for contentId: ${contentId}`);
+ // Same lookups as before (movie first, then series) but projected to the fields returned:
+ // core calls this per watch-history row and a full series document is ~40KB to decode.
+ const movieContentType = await this.videosRepository.getVideoContentType(contentId);
+ if (movieContentType) {
+ return { contentType: movieContentType as ContentInformationDto['contentType'] };
}
-
- if (!seriesContentData && !movieContentData) {
+ const seriesContentData = await this.seriesRepository.getSeriesStructure(contentId);
… (87 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| contentData CPU/request | 2.05 ms | 0.95 ms | −54% |
| Requests/s | 558 | 1,215 | |
| p99 | 60 ms | 27 ms |
- Output check
- 0 differences on 142 ids (movies, series, episodes, unknown ids).
9. Watch-history display: seasons only when needed — watch-history display
- Problem
- For every series parent the response carried every season's images and episode lists, but analytics only reads a season's images when the requested episode has none of its own (~40% of response bytes).
- Fix
- First query returns documents without seasons plus which requested ids each parent holds; a second query reads seasons (filtered to the one holding the episode) only for parents of an episode lacking images.
- Files
content.repository.ts
+ // Analytics reads a parent's seasons only to find the images of a requested episode whose own
+ // document has none. Season images were ~40% of this route's BSON, and requested episodes
+ // almost always carry their own images, so read seasons in a second query only for parents
+ // that hold such an episode. The first query returns, per document, which requested ids its
+ // seasons contain (small) instead of the seasons themselves.
+ const documents = await this.materializedViewRepository
+ .aggregate<WatchHistoryContentDisplay & { _eps?: string[] }>(
+ [
+ { $match: { contentId: { $in: contentIds } } },
+ { $set: { _l: this.localizedFor(requestedLanguage) } },
+ {
+ $project: {
+ _id: 0,
+ contentId: 1,
+ images: 1,
+ _eps: {
+ $setIntersection: [
+ {
+ $reduce: {
+ input: { $ifNull: ['$seasons', []] },
+ initialValue: [],
+ in: {
+ $concatArrays: [
+ '$$value',
+ { $ifNull: ['$$this.episodes.contentId', []] },
+ { $ifNull: ['$$this.episodes.id', []] },
+ ],
+ },
+ },
+ },
… (88 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| CPU/request, compressed | 8.26 ms | 4.62 ms | −44% |
| CPU/request, uncompressed | 3.49 ms | 2.86 ms | −18% |
| p99 | 160 ms | 100 ms |
- Output check
- 0 mismatches across 377,730 consumer rows built with analytics' exact mapping, including 78 episodes with no images of their own.
10. Pages: read each title once per page — page routes
- Problem
- Each module on a page read its own titles, so a title shown in several trays was decompressed and parsed again (home page: 396 tray items, 268 unique).
- Fix
- Share one read per id across the page's trays. Reel/promo modules keep their own copies because they write fields onto items.
- Files
pages.service.ts
+ // Reel/promo modules write promo-parent fields onto their items, so they never share item objects.
+ const memo =
+ module?.moduleType === ModuleTypes.STANDALONE_REEL_PLAYER || module?.settings?.contentType === ContentType.PROMO
+ ? undefined
+ : contentMemo;
const curatedItems = module?.settings?.curatedItems;
@@ -528,20 +544,30 @@ export class PagesService implements PagesServiceInterface {
await Promise.all([
- this.moviesService.getVideosByContentId(videoIds, language, platform, { limit: Infinity }),
- this.seriesService.getSeriesByContentId(seriesIds, language, platform, {
- limit: Infinity,
- joins: { recommendedContents: true, people: true },
- withAllEpisodeFields: true,
- }),
- isHoichoiFMEnabled ? this.audioService.getAudioByContentIds(audioIds, language, platform) : Promise.resolve([]),
+ this.fetchShared(memo, 'video', videoIds, (ids) =>
+ this.moviesService.getVideosByContentId(ids, language, platform, { limit: Infinity }),
+ ),
+ this.fetchShared(memo, 'series', seriesIds, (ids) =>
+ this.seriesService.getSeriesByContentId(ids, language, platform, {
+ limit: Infinity,
+ joins: { recommendedContents: true, people: true },
+ withAllEpisodeFields: true,
+ }),
+ ),
isHoichoiFMEnabled
- ? this.audioSeriesService.getAudioSeriesByContentId(audioSeriesIds, language, platform)
+ ? this.fetchShared(memo, 'audio', audioIds, (ids) =>
+ this.audioService.getAudioByContentIds(ids, language, platform),
+ )
… (104 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Page CPU/request | 236 ms | 163 ms | −31% |
- Output check
- Every old/new difference also appears between two runs of the unchanged code (shuffled recommendations, cache-state image variants); side-by-side capture: 0/23 URLs differ.
11. Pages: filter walk touches only what it changes — page routes
- Problem
filterContentPropertiesrebuilt every array and wroteseo = undefinedonto ~60k objects per page for non-web platforms.- Fix
- One in-place walk that only touches objects with
seo/localizedInfo; it still assignsundefinedrather than deleting (deleting made the later stringify 3× slower on Node 26). - Files
pages.service.ts
+ * Shared objects are safe because the per-module trimming and filterContentProperties are idempotent.
+ */
+ private async fetchShared<T extends { contentId?: string }>(
+ memo: ContentMemo | undefined,
+ kind: string,
+ ids: string[] | undefined,
+ load: (ids: string[]) => Promise<T[] | undefined>,
+ ): Promise<T[] | undefined> {
+ if (!memo || !ids) return load(ids);
+ const fresh = [...new Set(ids)].filter((id) => !memo.has(`${kind}:${id}`));
+ if (fresh.length > 0) {
+ const batch = load(fresh);
+ for (const id of fresh) memo.set(`${kind}:${id}`, batch);
+ }
+ const wanted = new Set(ids);
+ const batches = await Promise.all([...new Set(ids.map((id) => memo.get(`${kind}:${id}`)))]);
+ return (batches as (T[] | undefined)[]).flatMap((batch) => batch ?? []).filter((item) => wanted.has(item?.contentId));
+ }
+
/**
@@ -846,34 +895,31 @@ export class PagesService implements PagesServiceInterface {
private filterContentProperties(obj: any, platform: Platform, language: Language): any {
- if (typeof obj !== 'object' || obj === null) {
- return obj;
- }
- if (Array.isArray(obj)) {
… (52 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Page CPU/request | 163 ms | 156 ms | −4% |
| Pages total (10+11) | 230.6 ms | 157.3 ms | −32% |
| Requests/s | 6.6 | 9.5 |
- Output check
- Byte-identical bodies (unit test against the old code + side-by-side capture).
12. Logs nobody receives are not written — all routes, all services
- Problem
- Per-request warn/info lines (e.g.
getContents request received95k/hour) went through pino, a 14-regex PII sanitizer and OTel export, but the prod collector drops every INFO except[RES]. - Fix
- Remove the duplicate controller lines (the
[RES]line already carries query, status and error), move request-logger INFO lines to debug, and skip formatting when the level is off. - Files
contents.controller.ts,req-logger-middleware.ts+ 4 middlewares,hoichoi-logger.ts
@@ -64,3 +64,4 @@ export class RequestLoggerMiddleware implements NestMiddleware {
// for now keeping as is, as it was prior.
- this.logger.info(
+ // debug: the prod collector drops every INFO except [RES], which carries the same request context.
+ this.logger.debug(
`[REQ] ${req?.method} ${attributes.path}`,
@@ -37,2 +37,3 @@ export class HoichoiLogger implements HoichoiLoggerInterface {
debug(message: string, context?: LogContext): void {
+ if (!this.logger.logger.isLevelEnabled('debug')) return;
const parsed = this.parseContext(context);
@@ -42,2 +43,3 @@ export class HoichoiLogger implements HoichoiLoggerInterface {
trace(message: string, context?: LogContext): void {
+ if (!this.logger.logger.isLevelEnabled('trace')) return;
const parsed = this.parseContext(context);| Measure | Before | After | Change |
|---|---|---|---|
| Single title CPU/request, uncompressed | 3.01 ms | 2.43 ms | −19% |
| Single title, compressed | 4.29 ms | 3.69 ms | −14% |
- Output check
- 0 differences on 834 URLs (620 × 200, 214 × 404). 0 such lines reached prod SigNoz in 24h (32.7M rows), so nothing visible is lost.
13. Validation container lookup remembered — every validated route
- Problem
- class-validator asks Nest's container for every constraint class on every request; Nest throws
UnknownElementException(with a stack trace) for each one it doesn't own, then class-validator falls back. - Fix
- Same lookup and fallback, but the outcome per class is remembered, so the throw happens once per class instead of once per request.
- Files
apps/contents/src/main.ts
+ // class-validator asks the container for every constraint class on every validation. Nest's
+ // lookup throws UnknownElementException (with a stack trace) for each constraint AppModule
+ // doesn't own, then class-validator falls back to its own instance: ~2% of CPU on validated
+ // routes. Same lookup and fallback as before, but the outcome per class is remembered, so the
+ // throw happens once per class instead of once per request.
+ const appModuleRef = app.select(AppModule);
+ const resolvedConstraints = new Map<unknown, unknown>();
+ useContainer(
+ {
+ get<T>(someClass: { new (...args: unknown[]): T } | ((...args: unknown[]) => T)): T {
+ if (!resolvedConstraints.has(someClass)) {
+ let instance: unknown;
+ try {
+ instance = appModuleRef.get(someClass);
+ } catch (lookupError) {
+ instance = undefined;
+ }
+ resolvedConstraints.set(someClass, instance);
+ }
+ return resolvedConstraints.get(someClass) as T;
+ },
+ },
+ { fallback: true, fallbackOnErrors: true },
+ );
| Measure | Before | After | Change |
|---|---|---|---|
| Watch-list CPU/request | 3.65 ms | 3.56 ms | −2.5% |
- Output check
- 400/404/201 behaviour unchanged. (A first attempt using
app.get(..., {strict:false})activated a previously inert country-code validator and hung requests; it was caught by the A/B and replaced.)
14. Log sanitizer skips messages it cannot change — every log line, every service
- Problem
- Every log message went through 14 sequential redaction regexes (9.6 µs per message), including the one
[RES]line per request, which never contains a secret. - Fix
- One combined pre-test built from the same patterns: if none of the 14 could match, the message is returned as-is. Any message that might match still gets all 14 passes.
- Files
log-sanitizer.util.ts,log-sanitizer.fast-path.spec.ts
+// Every replace below needs at least one of these in its match (one alternative per regex, case-insensitive
+// like the ones that are). A message with none of them is returned unchanged by all 14 passes, so testing
+// this once lets clean messages (most request lines) skip the passes. Keep in sync with scrubText.
+const SECRET_KEYS =
+ 'authorization|auth[_-]?header|access[_-]?token|refresh[_-]?token|token|jwt|secret|private[_-]?key|hmac|signature|credential|password|passwd|otp|pin|cookie|session[_-]?(?:id|handle|key|token)|preauth(?:session)?[_-]?(?:id|handle)|code[_-]?id|userinputcode|verificationcode';
+const IDENTITY_KEYS =
+ 'user[_-]?id|device[_-]?(?:id|identifier)|phone[_-]?(?:number)?|mobile[_-]?(?:number)?|email(?:address)?|recipient(?:phone|email)?|to|identifier|normalized[_-]?identifier|phone[_-]?or[_-]?email|subscriber[_-]?(?:id|number)|msisdn|client[_-]?ip';
+const MAY_NEED_SCRUB = new RegExp(
+ [
+ 'bearer',
+ 'basic',
+ '[\\da-f]{8}-',
+ 'https?:',
+ '\\?',
+ 'eyj',
+ 'otp',
+ 'one[- ]time',
+ // the two assignment regexes: same keyword groups, then the same separator
+ `(?:${SECRET_KEYS}|${IDENTITY_KEYS})["']?\\s*[:=]`,
+ 'key',
+ '@',
+ '\\d[\\d ().-]{7,}\\d',
+ '\\d\\.\\d',
+ ].join('|'),
+ 'i',
+);
+
function scrubText(value: string): string {
+ if (!MAY_NEED_SCRUB.test(value)) return value;
return value| Measure | Before | After | Change |
|---|---|---|---|
| Sanitizer per message (prod shapes) | 9.6 µs | 1.8 µs | −81% |
| Watch-list CPU/request (with #15, #16) | 1.77 ms | 1.65 ms | −7% |
| Single title CPU/request (with #15, #16) | 2.32 ms | 2.22 ms | −4% |
- Output check
- 0 differences between old and new sanitizer over 300,138 inputs (real prod log bodies + fuzzed secrets, emails, phones, IPs, JWTs, URLs, UUIDs, key/value pairs). A spec pins both sides. Client-visible responses: 0 differences (side-by-side: 150 watch-list pages, 150 display, 150 contentData, 834 single-title URLs).
15. Disabled log levels skip context building — every debug/trace call
- Problem
- The contextual logger copied its context object before the level check in the layer below, so every disabled debug line still allocated.
- Fix
- Check the level first; the context is only built for records that will be written.
- Files
hoichoi-contextual-logger.ts,hoichoi-logger.ts,hoichoi-logger.interface.ts
@@ -39,2 +39,4 @@ export class HoichoiContextualLogger implements HoichoiLoggerInterface {
debug(message: string, context?: LogContext): void {
+ // Skip the context copy for a level that is off (debug lines run on every request).
+ if (this.baseLogger.isLevelEnabled?.('debug') === false) return;
this.baseLogger.debug(message, this.mergeContext(context));
@@ -42,2 +44,6 @@ export class HoichoiContextualLogger implements HoichoiLoggerInterface {
+ isLevelEnabled(level: 'debug' | 'trace'): boolean {
+ return this.baseLogger.isLevelEnabled?.(level) ?? true;
+ }
+
fatal(message: string, context?: LogContext): void {
@@ -51,2 +57,3 @@ export class HoichoiContextualLogger implements HoichoiLoggerInterface {
trace(message: string, context?: LogContext): void {
+ if (this.baseLogger.isLevelEnabled?.('trace') === false) return;
this.baseLogger.trace(message, this.mergeContext(context));| Measure | Before | After | Change |
|---|---|---|---|
| (measured together with #14 and #16) |
- Output check
- No output effect: disabled records were already dropped.
16. Series DRM filter skips episodes without renditions — watch-list, pages, single title
- Problem
- The per-platform rendition filter walked every episode of every series even for light variants whose episodes are trimmed to
{contentId, title}, and re-read and re-splitNON_DRM_PLATFORMSfrom the environment on every episode. - Fix
- Skip episodes whose
renditionsis undefined (only undefined; null keeps the old behaviour) and split the env value once per distinct value. - Files
content-series.service.ts,content-movies.service.ts
@@ -58,2 +58,14 @@ import {
type CachedVideo = ContentVideoResponse | NegativeCacheSentinel;
+
+// filterDRMRenditionsTemporarily runs per video and per series episode; split the env value only when it changes.
+let nonDrmRaw: string | undefined;
+let nonDrmList: string[] = [];
+function nonDrmPlatformList(): string[] {
+ const raw = process.env.NON_DRM_PLATFORMS;
+ if (raw !== nonDrmRaw) {
+ nonDrmRaw = raw;
+ nonDrmList = raw ? raw.split(',') : [];
+ }
+ return nonDrmList;
+}
type CachedVideoCard = WatchListContentDisplay | NegativeCacheSentinel;
@@ -303,3 +315,3 @@ export class ContentMoviesService implements ContentMoviesServiceInterface {
): RenditionsResponse[] {
- const nonDrmPlatforms = process.env.NON_DRM_PLATFORMS ? process.env.NON_DRM_PLATFORMS.split(',') : [];
+ const nonDrmPlatforms = nonDrmPlatformList();
const filteredVideoRenditions = renditions?.filter((rendition) => {
@@ -650,2 +650,5 @@ export class ContentSeriesService implements ContentSeriesServiceInterface {
season.episodes = season?.episodes?.map((episode) => {
+ // Light variants trim episodes to {contentId, title}: nothing to filter (the filter would only
+ // write renditions: undefined, which serialises away).
+ if (episode && episode.renditions === undefined) return episode;
episode.renditions = this.videosService.filterDRMRenditionsTemporarily(| Measure | Before | After | Change |
|---|---|---|---|
| (measured together with #14 and #15) |
- Output check
- Output identical (the filter only wrote
renditions: undefined, which serialises away); covered by the side-by-side checks under #14.
17. Pages: series trays read a pre-stripped projection — page routes
- Problem
- Shared page trays decoded the full series response (every episode with renditions, people, recommendations; ~198 KB raw) and then stripped most of it with
removeCuratedSeriesAndVideosNotRequiredData. Decoding was the largest slice of page CPU. - Fix
- Store the stripped result as field
seriestray:<lang>in the series' own Valkey hash. The strip moved intostripForPageTrayso both paths produce the same item. A miss loads the full variant fresh from Mongo (never from an older cache entry).refreshCachere-derives it on publish, and unpublish deletes the hash. Reel/promo modules still read the full series. - Files
content-series.service.ts,page-tray.util.ts(new),pages.service.ts,series.service.interface.ts,cache-keys.constants.ts
+ public async getSeriesTraysByContentId(
+ ids: string[],
+ language: Language,
+ platform: Platform,
+ ): Promise<ContentSeriesResponse[]> {
+ if (!ids?.length) return [];
+ const cacheSuffix = `${SERIES_TRAY_GROUPED_CACHE_TYPE_PREFIX}${language}`;
+ const cached = await this.cacheService.getorLoadManyExternalSWR<CachedSeries>(
+ ids.map((id) => ({ id, type: cacheSuffix })),
+ async (missed: CacheKeyInterface[]) => {
+ const full = await this.loadSeriesByContentIds(
+ `${SERIES_GROUPED_CACHE_TYPE_PREFIX}${language}:withPeople:withRecommendedContent:withAllEpisodeFields`,
+ missed.map((key) => key.id),
+ language,
+ platform,
+ PAGE_TRAY_SERIES_OPTIONS,
+ );
+ const loaded = full
+ .filter((entry) => entry.value && 'contentId' in entry.value)
+ .map((entry) => ({
+ id: entry.id,
+ type: cacheSuffix,
+ value: stripForPageTray(entry.value as ContentSeriesResponse) as ContentSeriesResponse,
+ }));
+ return withNotFoundMarkers(missed, loaded);
+ },
+ );
+ return this.applySeriesRenditionFilter(
+ withoutNotFoundMarkers(cached.map((key) => key.value)).filter(Boolean),
+ platform,
… (6 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Page CPU/request (compressed) | 153.2 ms | 98.7 ms | −36% |
| Requests/s | 10 | 14 | |
| Tray payload vs full | 198 KB | 32 KB | −84% |
| JSON.parse per series | 8.6× faster |
- Output check
- Byte-identical on all 23 page URLs (36.9 MB) side by side, twice, after resetting UAT's stale full-series fields (an older cache entry can differ from a fresh Mongo build; the tray is always built fresh).
18. Pages: movie trays read a pre-stripped projection — page routes
- Problem
- After #17 movies were the biggest decode on a page: each tray movie decoded the full response, then dropped
people[].relatedContentsand recommendation images/trailers. - Fix
- Same pattern as #17: field
videotray:<lang>in the movie's hash, built fresh on a miss, re-derived byrefreshCache, deleted on unpublish. The per-platform DRM rendition filter still runs after the read, as before. - Files
content-movies.service.ts,pages.service.ts,videos.service.interface.ts,cache-keys.constants.ts,index.ts
+ * Page-tray reads: the full response with stripForPageTray already applied, stored as 'videotray:<lang>' in
+ * the video's hash (unpublish deletes it, refreshCache re-derives it). Built from a fresh load, not from the
+ * 'video:<lang>' entry, so the tray never inherits a stale full entry. Renditions are filtered per platform
+ * after the read, exactly as getVideosByContentId does.
+ */
+ public async getVideoTraysByContentId(
+ ids: string[],
+ language: Language,
+ platform: Platform,
+ ): Promise<ContentVideoResponse[]> {
+ if (!ids?.length) return [];
+ const cacheSuffix = `${VIDEO_TRAY_GROUPED_CACHE_TYPE_PREFIX}${language}`;
+ const cached = await this.cacheService.getorLoadManyExternalSWR<CachedVideo>(
+ ids.map((id) => ({ id, type: cacheSuffix })),
+ async (missed: CacheKeyInterface[]) => {
+ const full = await this.loadVideosByContentIds(`video:${language}`, missed.map((key) => key.id), language, platform);
+ const loaded = full
+ .filter((entry) => entry.value && 'contentId' in entry.value)
+ .map((entry) => ({
+ id: entry.id,
+ type: cacheSuffix,
+ value: stripForPageTray(entry.value as ContentVideoResponse) as ContentVideoResponse,
+ }));
+ return withNotFoundMarkers(missed, loaded);
+ },
+ );
+ return this.applyRenditionsFilter(withoutNotFoundMarkers(cached.map((key) => key.value)).filter(Boolean), platform);
+ }
+
public filterDRMRenditionsTemporarily(
renditions: RenditionsResponse[],
video: ContentVideoResponse,
@@ -397,6 +429,7 @@ export class ContentMoviesService implements ContentMoviesServiceInterface {
const existingFields = await this.cacheService.listGroupedFields(contentId, ContentVideosCacheKey);
… (34 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Page CPU/request (compressed) | 80.95 ms | 68.38 ms | −15.5% |
| Requests/s | 18 | 22 |
- Output check
- Byte-identical on all 23 page URLs (34.7 MB) side by side, twice, after resetting UAT's
video:*fields.
19. Mongo: one Db handle per client instead of one per query — contentData, gates, every TypeORM read
- Problem
- TypeORM's Mongo query runner calls
client.db(name)on every query (MongoQueryRunner.getCollection). Each call copies the client options and constructs a newDb(option filtering, read/write concern, BSON options). The profiles showeddb+filterOptionsat 1.3–2.6% of busy time on gates and contentData. - Fix
- At startup, wrap
MongoClient.prototype.dbso a call without per-call options returns the sameDbfor that client and name. ADbtakes its settings from the client when built and our code never changes them later; calls that pass options still build a new one. - Files
reuse-mongo-db-handles.ts(new) + spec,apps/contents/src/main.ts,index.ts
+ * TypeORM's Mongo driver calls `client.db(name)` on every query (MongoQueryRunner.getCollection), and each call
+ * copies the client options and builds a new Db (option filtering, read/write concern, BSON options). A Db is
+ * an immutable handle derived from the client's options at construction, so the handle for a given name is
+ * reused when db() is called without per-call options; calls with options keep the old behaviour.
+ */
+export function reuseMongoDbHandles(): void {
+ const proto = MongoClient.prototype as MongoClient & { [PATCHED]?: boolean };
+ if (proto[PATCHED]) return;
+ const original = proto.db;
+ const handles = new WeakMap<MongoClient, Map<string | undefined, ReturnType<MongoClient['db']>>>();
+ proto.db = function db(this: MongoClient, dbName?: string, options?: Parameters<MongoClient['db']>[1]) {
+ if (options !== undefined) return original.call(this, dbName, options);
+ let byName = handles.get(this);
+ if (!byName) handles.set(this, (byName = new Map()));
+ let handle = byName.get(dbName);
+ if (!handle) byName.set(dbName, (handle = original.call(this, dbName)));
+ return handle;
+ };
+ proto[PATCHED] = true;
+}| Measure | Before | After | Change |
|---|---|---|---|
| contentData CPU/request (6 paired rounds) | 1.18 ms | 1.11 ms | −6% (lower in 6/6 rounds) |
| Gates CPU/request (6 paired rounds) | 1.35 ms | 1.32 ms | −2.9% (lower in 6/6 rounds) |
- Output check
- 0 differences on 450 side-by-side requests (150 each: contentData, gates, display). The lab's background load moved between rounds, so the full-spread rule calls it noise; each round's pair, run back to back, came out lower every time.
20. Replies stay on V8's fast JSON.stringify path — page routes, watch-history display, platform
- Problem
- Node 26's fast
JSON.stringifyis all-or-nothing: one ineligible object anywhere in the reply sends the whole serialisation to the slow path. Page replies carried 10 class instances (module/settings/seo shells fromplainToInstance); a first fix copied them with{...x}, which tsc compiles toObject.assign({}, x)for our es2017 target, and V8 turns that copy into a dictionary-mode object at 20+ keys (SettingsResponse has exactly 20). Display useddelete, and the 541-key platform messages object comes out ofJSON.parsein dictionary mode. - Fix
- Copy only those shells with
Object.fromEntries(same own keys, same order), assignundefinedinstead ofdelete, rebuild the messages object on a new response object. - Files
pages.service.ts,content.repository.ts,content.service.ts,pages.plain-shells.spec.ts
+ // object and pushes the whole response off V8's fast JSON.stringify path (~1.5-2.7x slower).
+ for (const document of documents) document._eps = undefined;
if (!parentIds.length) {
return documents;
}
@@ -70,6 +70,25 @@ import { stripForPageTray } from '../utilities/page-tray.util';
type ContentMemo = Map<string, Promise<unknown[] | undefined>>;
+/**
+ * V8's fast JSON.stringify path (Node 25+) is all-or-nothing: one ineligible object anywhere in the tree sends the
+ * whole serialisation to the slow path (~3.5x slower on a 1.1 MB page). Ineligible: any non-Object.prototype
+ * prototype (the PageModuleResponse/SettingsResponse/PagesResponse/PageSeoResponse shells from plainToInstance) and dictionary-mode
+ * objects. Copy only those shells with Object.fromEntries: `{ ...x }` is compiled to `Object.assign({}, x)` for our
+ * es2017 target, and V8 turns that copy into a dictionary-mode object at 20+ keys (SettingsResponse has 20), which is
+ * just as slow. Own enumerable keys in the same order, so the JSON is byte-identical (no getters/toJSON here).
+ */
+export function toPlainShells<T>(value: T): T {
+ if (Array.isArray(value)) return value.map((item) => toPlainShells(item)) as unknown as T;
+ if (value === null || typeof value !== 'object' || Object.getPrototypeOf(value) === Object.prototype) return value;
+ const plain: Record<string, unknown> = Object.fromEntries(Object.entries(value));
+ // Class-instance children of the shells: module settings, page seo and its openGraphDetails (PageSeoResponse).
+ for (const key of ['settings', 'modules', 'seo', 'openGraphDetails'] as const) {
+ if (plain[key] !== undefined) plain[key] = toPlainShells(plain[key]);
+ }
+ return plain as T;
+}
+
@Injectable()
export class PagesService implements PagesServiceInterface {
// Caching in local memory for 5 minutes
@@ -210,7 +229,7 @@ export class PagesService implements PagesServiceInterface {
pageResponse.modules = modules;
pageResponse = this.filterContentProperties(pageResponse, platform, language);
… (15 more lines in commit 01a5b1a9cd)| Measure | Before | After | Change |
|---|---|---|---|
| Pages CPU/request | 61.30 ms | 55.70 ms | −9.1% |
| Watch-history display CPU/request | 2.27 ms | 2.06 ms | −9.3% |
| 1.1 MB page stringify | 6.1 ms | ~1.6 ms |
- Output check
- Byte-identical side by side on all routes: 9,227 requests (23 pages, 3,000 single titles, 2,000 entitlement, 2,000 contentData, display, gates, watch-list), 0 mismatches. Verified by an independent re-check of the raw files.
Tests
- contents: 31 suites / 252 tests pass (243 at the start; new specs for watch-list display, video cards, video trays, page read-once, entitlement detection).
- hoichoi-common: logger level guard, device detector helper, anon token key, internal identity header specs pass.
- analytics, grameen, support-api, posthog-ingester-proxy, coreapis: type-check with 0 errors.
- Pre-existing failures left as found (fail on
origin/maintoo):exception-response-enrichmentfull_url expectations,request-logger.utilsmissing exports, redis-cache/cache-manager specs.