wezm/youtube-to-opml
1
0
Fork 1
Code Issues Pull requests 1 Projects Releases Packages Wiki Activity

main #1

Open
vintagefuture wants to merge 2 commits from vintagefuture/youtube-to-opml:main into main
pull from: vintagefuture/youtube-to-opml:main
merge into: wezm:main
Conversation 0 Commits 2 Files changed 2 +2 -3
2 changed files with 2 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
Makefile
View file

@ -10,8 +10,7 @@ fetch: subscriptions.curl
curl --location --output-dir html --create-dirs --rate 1/s --config subscriptions.curl curl --location --output-dir html --create-dirs --rate 1/s --config subscriptions.curl
channel-json: subscriptions.txt channel-json: subscriptions.txt
# nproc is not portable :-/ cat subscriptions.txt | xargs -n1 --verbose ./generate-json-opml
wezm commented 2024-11-11 04:46:17 +00:00
Review
Copy link

The parallelism was important to me as I had hundreds of subscriptions to process and a 16c/32t CPU to keep busy so I'd like to keep it.

the --max-procs flag, which is not supported on macOs's version of xargs.

--max-procs can be replaced with -P which is supported by the BSD xargs used on macOS as well as GNU.

we could do something like NUM_PROCS=$(shell uname | grep -q "Darwin" && sysctl -n hw.ncpu || nproc)

That seems mostly fine. Suggest changing it to use uname -s though.

The parallelism was important to me as I had hundreds of subscriptions to process and a 16c/32t CPU to keep busy so I'd like to keep it. > the --max-procs flag, which is not supported on macOs's version of xargs. `--max-procs` can be replaced with `-P` which is supported by the BSD xargs used on macOS as well as GNU. > we could do something like NUM_PROCS=$(shell uname | grep -q "Darwin" && sysctl -n hw.ncpu || nproc) That seems mostly fine. Suggest changing it to use `uname -s` though.
xargs -n1 --max-procs=$$(nproc) --arg-file subscriptions.txt --verbose ./generate-json-opml
# turn all the channel json files into an OPML file # turn all the channel json files into an OPML file
subscriptions.opml: subscriptions.opml:

2
generate-json-opml
View file

@ -17,4 +17,4 @@ JSON_TITLE=$(json_escape "$TITLE")
JSON_XML_URL=$(json_escape "$XML_URL") JSON_XML_URL=$(json_escape "$XML_URL")
JSON_URL=$(json_escape "$URL") JSON_URL=$(json_escape "$URL")
printf '{"title": %s, "xmlUrl": %s, "htmlUrl": %s}\n' "$JSON_TITLE" "$JSON_XML_URL" "$JSON_URL" > json/"$NAME".json mkdir -p json && printf '{"title": %s, "xmlUrl": %s, "htmlUrl": %s}\n' "$JSON_TITLE" "$JSON_XML_URL" "$JSON_URL" > json/"$NAME".json