51 words
1 minute
Running pyspider on Windows 11 with Docker
There were issues installing pyspider on Windows 11, with multiple errors occurring.
I found that the official website offers a Docker-based installation method.
Directly via Docker
# mysqldocker run --name mysql -d -v /data/mysql:/var/lib/mysql -e MYSQL_ALLOW_EMPTY_PASSWORD=yes mysql:latest# rabbitmqdocker run --name rabbitmq -d rabbitmq:latest
# phantomjsdocker run --name phantomjs -d binux/pyspider:latest phantomjs
# result workerdocker run --name result_worker -m 128m -d --link mysql:mysql --link rabbitmq:rabbitmq binux/pyspider:latest result_worker# processor, run multiple instance if needed.docker run --name processor -m 256m -d --link mysql:mysql --link rabbitmq:rabbitmq binux/pyspider:latest processor# fetcher, run multiple instance if needed.docker run --name fetcher -m 256m -d --link phantomjs:phantomjs --link rabbitmq:rabbitmq binux/pyspider:latest fetcher --no-xmlrpc# schedulerdocker run --name scheduler -d --link mysql:mysql --link rabbitmq:rabbitmq binux/pyspider:latest scheduler# webuidocker run --name webui -m 256m -d -p 5000:5000 --link mysql:mysql --link rabbitmq:rabbitmq --link scheduler:scheduler --link phantomjs:phantomjs binux/pyspider:latest webuiUsing docker-compose
services: phantomjs: image: binux/pyspider:latest command: phantomjs result: image: binux/pyspider:latest external_links: - mysql - rabbitmq command: result_worker processor: image: binux/pyspider:latest external_links: - mysql - rabbitmq command: processor fetcher: image: binux/pyspider:latest external_links: - rabbitmq links: - phantomjs command : fetcher scheduler: image: binux/pyspider:latest external_links: - mysql - rabbitmq command: scheduler webui: image: binux/pyspider:latest external_links: - mysql - rabbitmq links: - scheduler - phantomjs command: webui ports: - "5000:5000"Then just run:
docker-compose up -d
After running successfully, if you visit http://localhost<5000>5000>/ and see the content below, it indicates that pyspider is running successfully.

Share
If this article helped you, please share it with others!
Running pyspider on Windows 11 with Docker
https://dreaife.tokyo/en/posts/docker-pyspider-win/ Some information may be outdated
Sunflower: A Coming-of-Age Story from Child to Adult
About Errors When Using pandas.to_datetime with Different Time Formats
Related Posts Smart
1
Python Web Crawler Environment Setup
spider Setting up a Python web crawler environment includes installing Python 3, request libraries (such as requests and selenium), parsing libraries (such as lxml and beautifulsoup4), databases (such as MySQL and MongoDB), storage libraries (such as PyMySQL and PyMongo), web libraries (such as Flask and Tornado), app crawling tools (such as mitmproxy and appium), and crawler frameworks (such as pyspider and scrapy). Installation commands and notes for each library are provided in detail.
2
Getting Started with Docker
infra Docker is a technology for solving microservice deployment problems by packaging applications and their dependencies into isolated containers, avoiding inconsistent environments and dependency conflicts. Compared with virtual machines, Docker starts faster and uses fewer resources. Its architecture includes images and containers, and users can share and obtain images through Docker Hub. Basic operations include creating and managing images and containers and using volumes for data persistence and host-container decoupling. Docker Compose can simplify distributed application deployment.
3
A Summary of My Thoughts on the Market on X
market Drawing on complex systems and game theory, this post unpacks the underlying logic of market dynamics, emotional reflexivity, and the consensus underpinning monetary value, helping readers look beyond short-term volatility and develop a deeper understanding of market expectations and inflation transmission.
4
Basic On-Chain Operations for EOA Wallets
WEB3 Quickly master the core on-chain operations of Web3 wallets. Covers the principles behind creating EOA and HD wallets, mnemonic generation and the BIP derivation process, an in-depth look at SIWE and EIP-712 on-chain verification, and the full lifecycle of transaction construction, signing, and broadcasting to help you understand the underlying logic of wallets.
5
About an EOA Wallet Signature Verification and Related Content
WEB3 A developer-focused guide to EOA wallet signature verification, covering secp256k1, ECDSA r/s/v signatures, public key recovery, keccak-256 address derivation, and how SIWE proves wallet ownership without exposing the private key.
Random Posts Random
1
Personal Online Scam 01: Scam Analysis
life 2024-08-19
2
Blog Migration - Mizuki Configuration Records
infra 2026-02-22
3
missing-semester-class01
cs-base 2023-01-11
4
Dreams and Hope Overflowing, Self-Forgiveness and Reverie — Sakura, Moyu.
gal 2022-09-18
5
Angular-Based Animation Showcase Website with Login and Registration (Cognito)
PROJECT 2024-11-12





