# Complete Guide: How to Scrape note.com Using note.com Hashtag Scraper This complete guide to [note.com](http://note.com) Hashtag Scraper shows how to scrape Japanese articles from [note.com](http://note.com) with Python through the Apify API, and get back one typed row per article: title, URL, Japan-time publication date, JPY price, free, paid and membership flags, likes, and the creator's handle and X account. With article pages on, rows also carry the readable text, paywalled character count and the creator's public profile. [note.com](http://note.com) is a Japanese creator and blog platform. Two things make a home-made crawler awkward. First, its robots.txt disallows the JSON API (`/api/*`) and keyword search (`/search`), so a hashtag page is the unit of reading and each one embeds a single page of articles. Second, a tag page sometimes comes back as HTTP 200 without its article list, so trusting the status code gets no articles from that page. You will configure the [**note.com**](http://note.com) **Hashtag Scraper** Apify Actor (`reventadata/note-scraper`), run it from Python, read its rows and use its filters. Prerequisites: Python 3.11+, the `apify-client` package and an Apify API token. One sitting. The sample row in this guide is a sample from a capture dated 2026-10-22, parsed with the Actor's own parser; this guide quotes no run of its own. --- ## TL;DR In this tutorial, you'll learn how to scrape [note.com](http://note.com) by hashtag with the [note.com](http://note.com) Hashtag Scraper Apify Actor from Python. Key steps: install `apify-client`, export `APIFY_API_TOKEN`, put your tags in `hashtags`, pick a `mode` (`popular`, `new` or `both`), optionally turn on `includeDetails`, call `reventadata/note-scraper`, and read the rows from the run's default dataset. Filters for keywords, free articles, likes and publication date drop rows before they are saved. Expected outcome: a JSON list of articles with prices, paywall flags and creator data, ready for a spreadsheet or database. The Actor: [apify.com/reventadata/note-scraper](https://apify.com/reventadata/note-scraper). --- ## Prerequisites and setup You will need: 1. Python 3.11+ (`apify-client` 3.3.0 requires it). 2. The Apify API client for Python. 3. An Apify account. Apify Console is Apify's web interface; your API token is on its **API & Integrations** page. The platform documentation starts at [docs.apify.com](https://docs.apify.com/). Install the client: ```bash pip install "apify-client~=3.0" ``` The token is a secret. Apify's docs advise giving each token a description and never using one token for several services. The code in this guide reads it from the environment, so it never sits in a script or a commit. Set the token in your environment: ```bash export APIFY_API_TOKEN="your_apify_token_here" ``` --- ## Understanding [note.com](http://note.com) Hashtag Scraper ### What it does and how it works 1. **Two URL shapes.** For each tag the Actor opens `https://note.com/tag/` (the popular list) and, with `mode` `new` or `both`, the same URL with `?mode=new`. With `includeDetails`, it also opens each article at `https://note.com//n/`. It never calls `/api/*` or `/search`. 2. **Transport.** Pages go through Apify residential proxy in Japan by default, the transport that returned full pages in testing on 2026-10-22; without a proxy or on datacenter exits the site sometimes answered the empty shell. 3. **Parsing.** The articles sit in the page's embedded Next.js data. The Actor counts the article objects it parsed: zero on a tag page means the shell, and it retries on a fresh exit. 4. **Filtering and saving.** Your filters run on each row; kept rows go to the dataset. ### Limitations - One page per tag and list: 20 articles on tags served as a search list, about 40-50 on large tags served as a timeline list. Related tags are the way past it. - No keyword search; `keywords` filters the rows the tag pages return. - A creator on a custom domain gets listing fields only. No emails are extracted. --- ## Step-by-step walkthrough ### Step 1: Initialize the client and define inputs The main input is `hashtags`: tags as written on [note.com](http://note.com) (`AI`, `#日記`) or tag URLs such as `https://note.com/tag/ChatGPT`. `mode` is `both` (default), `popular` or `new`. `expandRelatedTags` (0 to 10, default 0) also reads that many of the related keywords each seed tag's page lists. `maxItems` (default 200) caps saved rows. This setup reads `AI` (popular and new), opens each article page and saves at most 10 rows: ```python import os from apify_client import ApifyClient # 1. Initialize ApifyClient with the token from the environment client = ApifyClient(os.environ["APIFY_API_TOKEN"]) # 2. Define the input: one hashtag, both lists, article pages on, at most 10 rows run_input = { "hashtags": ["AI"], "mode": "both", "includeDetails": True, "maxItems": 10, } ``` No proxy setting is needed: without one, the Actor uses Apify residential proxy, country JP. Two more configurations, as JSON input: - **Newest posts under two tags:** `{"hashtags": ["副業", "ChatGPT"], "mode": "new", "maxItems": 50}`. - **Wider net from one seed:** `{"hashtags": ["AI"], "mode": "popular", "expandRelatedTags": 5, "maxItems": 100}`. The `AI` page captured on 2026-10-22 listed `AGI` (19,799 notes), `AI Agent` (1,215) and `AI Act` (604) among its 10 related keywords. ### Step 2: Configure filters Every filter runs on the rows after the Actor reads them: - `keywords` / `excludeKeywords`: substrings, case- and width-insensitive (`ＡＩ` matches `AI`). Without details they check the title; with details also the text, description and creator bio. - `freeOnly`: keep only articles the page marks free, with no price and no membership gate. - `minLikes`: at least this many likes. - `publishedAfter`: `YYYY-MM-DD`, read as midnight Japan time, or an ISO timestamp. A `mode` outside the three values is refused before the run starts; a `publishedAfter` that is not a date stops the run with an error at its start. ### Step 3: Optional enrichment with article pages `includeDetails: true` opens each article page and adds `bodyHtml`, `bodyText`, `bodyChars`, `paywalledChars`, `categories`, `hashtags`, `anonymousLikeCount` and the creator's `creatorFollowers`, `creatorFollowing`, `creatorNoteCount`, `creatorProfile`, `creatorExternalLinks`, `creatorSocials`, `creatorCreatedAt` and `contactUrls`. It costs one extra request per article, so runs are slower. On a paid article page captured 2026-10-22 (price 770 JPY), the parser returned 1,397 readable characters, 2,997 behind the paywall, the categories `テクノロジー` and `IT`, and the creator's Instagram and YouTube links. ### Step 4: Run the Actor `.call()` starts the run and waits for it to finish: ```python # 3. Start the Actor run and wait for it to finish print("Starting note.com extraction...") run = client.actor("reventadata/note-scraper").call(run_input=run_input) if run is None or run.status != "SUCCEEDED": status = run.status if run else "unknown" raise SystemExit(f"Run did not succeed (status: {status}).") # 4. Fetch the saved articles items = client.dataset(run.default_dataset_id).list_items().items print(f"Retrieved {len(items)} articles.\n") ``` ### Step 5: Process the rows ```python for item in items: print(item["title"]) print(f" URL: {item['url']}") print(f" Creator: {item.get('creatorName')} (@{item.get('creatorUrlname')})") if item.get("isFree"): print(" Free to read") else: print(f" Price: {item.get('price')} JPY, membership: {item.get('hasSubscription')}") print(f" Likes: {item.get('likeCount')}, published: {item.get('publishedAt')}") if item.get("detailsStatus") == "ok": print(f" Followers: {item.get('creatorFollowers')}, paywalled chars: {item.get('paywalledChars')}") print("-" * 40) ``` ### Step 6: Inspect a real row A sample from a capture dated 2026-10-22 (the `AI` tag's popular list plus that article's page), parsed with the Actor's own parser. Null and empty fields, `bodyHtml`, the two image URLs and `scrapedAt` (added at save time) are left out: ```json { "id": 185275155, "key": "n6ccbac30a263", "title": "【朝メモ】 Meta 新AIモデル発表", "url": "https://note.com/goto_finance/n/n6ccbac30a263", "tag": "AI", "listMode": "popular", "publishedAt": "2026-10-13T06:59:35.000+09:00", "price": 0, "currency": "JPY", "isFree": false, "hasSubscription": true, "isPaywalled": true, "canRead": false, "likeCount": 750, "commentCount": 10, "imageCount": 3, "format": "4.0", "creatorUrlname": "goto_finance", "creatorName": "後藤達也", "creatorUrl": "https://note.com/goto_finance", "creatorTwitter": "goto_finance", "bodyText": "おはようございます。\n米国はコロンバスデーで、債券市場は休場でした。株式市場ではメモリ関連株が下落したものの、S&P500はほぼ横ばいでした。\n\n今朝はMetaの新しいAIと、今晩から本格化する米企業決算についてまとめます。\n◆ Meta、AI戦略に変化", "bodyChars": 127, "paywalledChars": 1022, "anonymousLikeCount": 8, "creatorFollowers": 94430, "creatorFollowing": 19, "creatorNoteCount": 1136, "creatorProfile": "経済ニュースを「わかりやすく、おもしろく」。経済や投資になじみのない方を念頭に、偏りのない情報発信を目指しています。国民の金融リテラシーの健全な向上に少しでも貢献できればと思っています。YouTube、Twitter、noteが3本柱。プラットフォーム間のシナジーも探ります。", "creatorSocials": { "twitter": "goto_finance" }, "creatorCreatedAt": "2022-03-06T04:00:44.000+09:00", "detailsStatus": "ok" } ``` ### Step 7: Export Apify datasets export as JSON, CSV, XML, Excel, HTML table, RSS or JSONL, from Console or through the API's `format` parameter. --- ## Working with the API The Python client gives two ways to start this Apify Actor. `client.actor("reventadata/note-scraper").call(run_input=run_input)` starts the run and waits for it to finish before returning the run object, or `None`; that is what the code above uses. For long runs or a job queue, `.start(run_input=run_input)` returns the run object at once, and you read it back later with its ID. Authentication is the API token in `APIFY_API_TOKEN`; keep it out of source control. Once a run finishes, `client.dataset(run.default_dataset_id).list_items().items` returns the rows in one call, and `iterate_items()` pages through them for larger datasets. `iterate_items()` also takes a `fields` argument, so you can skip `bodyHtml` when you only need counts. For event-driven pipelines, attach a webhook for the `ACTOR.RUN.SUCCEEDED` event. Its payload's `resource` is the response of the Get Actor run API endpoint, so it carries `defaultDatasetId`, and your handler can fetch the rows without polling. Error handling: check `run.status` as the code does. The status is `SUCCEEDED`, `FAILED`, `TIMED-OUT` or `ABORTED`. A run that saves no rows ends `FAILED`, whether it cannot read a tag page, every tag is unknown or your filters dropped every article. When a run fails, read its log from Python with `client.log(run.id).get()`; the summary line, `Run finished: N tag pages and N article pages fetched, N rows pushed. Tally: ...`, counts each filter's drops. Rate and politeness: the Actor requests only the robots-allowed tag and article pages: one page per tag and list (again only when the shell came back), plus one article page per article that passes the listing filters when details are on. --- ## Advanced usage patterns ### Batch processing multiple tags Pass several tags in `hashtags`, for example `["AI", "ChatGPT", "生成AI"]`. They are read in the order given within one run and fill one dataset. An article kept under one tag or list is not saved again under a later one, and an article a filter rejected is not fetched again. The run stops once `maxItems` is reached. ### Scheduled runs Save the input as a task and add an Apify schedule. Schedules run an Actor or task on a cron expression; `0 8 * * *` is every day at 8 AM. Pair it with `mode: "new"` and a `publishedAfter` date to keep each run to recent posts. ### Monitoring Combine the schedule with a webhook on `ACTOR.RUN.SUCCEEDED` to load each new dataset, and treat a `FAILED` run as a signal: with this Actor it means no rows were saved: a tag page could not be read, every tag was unknown (a mistyped tag) or your filters dropped every article. ### Cost and volume control `maxItems` is the bound on saved rows, and `includeDetails` is the switch that multiplies requests: off, one request per tag page; on, one more per article that passes the listing filters. Filters that only need listing fields (`freeOnly`, `minLikes`, `publishedAfter`, an `excludeKeywords` hit in the title) drop rows before any article page is opened. To scale out, add tags or `expandRelatedTags` rather than expecting more from one tag page. --- ## Understanding your data Key fields and how to read them: - `price`, `isFree`, `hasSubscription`: `price` is the single-purchase price in JPY, 0 when none. `price` 0 is not free: on the search list `isFree` is the page's own flag, and on the timeline list it means price 0 with no membership gate. In the 20-article `AI` popular capture, none was free, 19 were membership articles, and 15 had `price` 0. - `isPaywalled`, `canRead`: whether a logged-out visitor can read the whole article. - `bodyChars`, `paywalledChars`: readable and paywalled characters. The sample has 127 and 1,022. - `commentCount`, `imageCount`: null on the timeline list, which carries likes only. Both `副業` captures (47 and 40 articles) had no comment counts. - `creatorFollowers`: null when the creator hides follow counts. - `hashtags`: listed on some article pages only. Of four article pages captured 2026-10-22, two listed hashtags (3 and 2) and two listed none. - `publishedAt`: ISO 8601 with the page's `+09:00` offset. --- ## Troubleshooting guide - **Run FAILED with zero rows:** every tag was unknown on [note.com](http://note.com) (HTTP 404), a tag page could not be read, or your filters dropped every article; check the tag spelling and the filters. A single unknown tag among several is logged and skipped. - **Run stops at its start:** no entry in `hashtags` was a tag or a [note.com](http://note.com) tag URL, or `publishedAfter` is not a date. - **Input refused before the run starts:** `mode` is not `both`, `popular` or `new`, or `expandRelatedTags` is outside 0 to 10. - **Only 20 rows from a big tag:** one page per tag and list is all the site's robots-allowed pages give. Use `mode: "both"` and `expandRelatedTags`. - `detailsStatus: "redirect"`**:** the creator publishes on a custom domain; the row keeps its listing fields. - **Few or no free articles:** on the `AI` popular list captured 2026-10-22, none of the 20 was free. Check `isFree`, not `price`. - **Reading the log:** `client.log(run.id).get()`; the tally in the summary line counts `tag_not_found`, `not_free`, `too_few_likes`, `too_old`, `keyword_mismatch` and the rest. For help, use the Actor's [Store page](https://apify.com/reventadata/note-scraper). --- ## Best practices 1. **Pick tags, not keywords.** A hashtag is the unit; `keywords` only filters what the tag pages return. 2. **Always set** `maxItems`**.** It bounds the run. 3. **Start small.** `maxItems: 10` until your code handles the fields. 4. **Turn on** `includeDetails` **only when you need text or creator profiles.** It adds one request per article. 5. **Mind freshness.** The newest list moves fast: all 19 articles in the `AI` newest capture were posted within two minutes. Schedule reruns instead of reusing old datasets. 6. **Handle failure explicitly.** Check `run.status` before reading rows, and read the log's summary line before trusting a short dataset. --- ## Pricing For current pricing and details, see the [Store page](https://apify.com/reventadata/note-scraper). --- ## Conclusion and next steps You configured [note.com](http://note.com) Hashtag Scraper, ran it from Python and read typed article rows with JPY prices, paywall flags and creator data, plus filters that run before a row is saved. Next: schedule a tag, attach a webhook, or turn on `includeDetails` for creator profiles. Try [note.com](http://note.com) Hashtag Scraper yourself and let me know what you build! *About the author: reventadata builds and maintains marketplace and content-platform scrapers on the Apify Store.* --- ## Resources - [note.com Hashtag Scraper on Apify Store](https://apify.com/reventadata/note-scraper) - [Apify Python client documentation](https://docs.apify.com/api/client/python/docs) - [Apify schedules](https://docs.apify.com/actors/running/schedules) - [Webhook actions](https://docs.apify.com/integrations/webhooks/actions)