⊟
165 KB
/srv/reproducible-results/rbuild-debian/r-b-build.mqQUAmdV/b1/python-scrapy_2.8.0-2_amd64.changes vs.
/srv/reproducible-results/rbuild-debian/r-b-build.mqQUAmdV/b2/python-scrapy_2.8.0-2_amd64.changes
⊟
295 B
Files
    
Offset 1, 3 lines modifiedOffset 1, 3 lines modified
  
1 ·e867b7a795e9a010172a75e0bbd68880·793284·doc·optional·python-scrapy-doc_2.8.0-2_all.deb1 ·6c7251b865045317e3c35dab968950bc·793288·doc·optional·python-scrapy-doc_2.8.0-2_all.deb
2 ·3c4de3d745944907c9555e5faf1feb1e·246168·python·optional·python3-scrapy_2.8.0-2_all.deb2 ·3c4de3d745944907c9555e5faf1feb1e·246168·python·optional·python3-scrapy_2.8.0-2_all.deb
⊟
165 KB
python-scrapy-doc_2.8.0-2_all.deb
⊟
370 B
file list
    
Offset 1, 3 lines modifiedOffset 1, 3 lines modified
1 -rw-r--r--···0········0········0········4·2023-03-26·13:57:50.000000·debian-binary1 -rw-r--r--···0········0········0········4·2023-03-26·13:57:50.000000·debian-binary
2 -rw-r--r--···0········0········0·····4588·2023-03-26·13:57:50.000000·control.tar.xz2 -rw-r--r--···0········0········0·····4592·2023-03-26·13:57:50.000000·control.tar.xz
3 -rw-r--r--···0········0········0···788504·2023-03-26·13:57:50.000000·data.tar.xz3 -rw-r--r--···0········0········0···788504·2023-03-26·13:57:50.000000·data.tar.xz
⊟
98.0 B
control.tar.xz
⊟
70.0 B
control.tar
⊟
48.0 B
./md5sums
⊟
30.0 B
./md5sums
Files differ
⊟
164 KB
data.tar.xz
⊟
164 KB
data.tar
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/404.html
    
Offset 137, 15 lines modifiedOffset 137, 15 lines modified
137 ···········</div>137 ···········</div>
138 ··········</div>138 ··········</div>
139 ··········<footer>139 ··········<footer>
  
140 ··<hr/>140 ··<hr/>
  
141 ··<div·role="contentinfo">141 ··<div·role="contentinfo">
142 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.142 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
143 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.143 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
144 ······</span></p>144 ······</span></p>
145 ··</div>145 ··</div>
  
146 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a146 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
147 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>147 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
148 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.148 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
545 B
html2text {}
    
Offset 62, 9 lines modifiedOffset 62, 9 lines modified
62 ···Scrapy62 ···Scrapy
63 ····*·Page·not·found63 ····*·Page·not·found
64 ===============================================================================64 ===============================================================================
65 ******·Page·not·found·******65 ******·Page·not·found·******
66 Unfortunately·we·couldn't·find·the·content·you·were·looking·for.66 Unfortunately·we·couldn't·find·the·content·you·were·looking·for.
  
67 ===============================================================================67 ===============================================================================
68 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.68 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
69 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.69 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.11 KB
./usr/share/doc/python-scrapy-doc/html/README.html
    
Offset 191, 15 lines modifiedOffset 191, 15 lines modified
191 ···········</div>191 ···········</div>
192 ··········</div>192 ··········</div>
193 ··········<footer>193 ··········<footer>
  
194 ··<hr/>194 ··<hr/>
  
195 ··<div·role="contentinfo">195 ··<div·role="contentinfo">
196 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.196 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
197 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.197 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
198 ······</span></p>198 ······</span></p>
199 ··</div>199 ··</div>
  
200 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a200 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
201 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>201 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
202 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.202 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
501 B
html2text {}
    
Offset 96, 9 lines modifiedOffset 96, 9 lines modified
96 To·compile·the·documentation·to·HTML·run·the·following·command:96 To·compile·the·documentation·to·HTML·run·the·following·command:
97 tox·-e·docs97 tox·-e·docs
98 Documentation·will·be·generated·(in·HTML·format)·inside·the·.tox/docs/tmp/html98 Documentation·will·be·generated·(in·HTML·format)·inside·the·.tox/docs/tmp/html
99 dir.99 dir.
  
  
100 ===============================================================================100 ===============================================================================
101 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.101 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
102 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.102 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.1 KB
./usr/share/doc/python-scrapy-doc/html/_modules/index.html
    
Offset 205, 15 lines modifiedOffset 205, 15 lines modified
205 ···········</div>205 ···········</div>
206 ··········</div>206 ··········</div>
207 ··········<footer>207 ··········<footer>
  
208 ··<hr/>208 ··<hr/>
  
209 ··<div·role="contentinfo">209 ··<div·role="contentinfo">
210 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.210 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
212 ······</span></p>212 ······</span></p>
213 ··</div>213 ··</div>
  
214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
472 B
html2text {}
    
Offset 130, 9 lines modifiedOffset 130, 9 lines modified
130 ····*·scrapy.utils.defer130 ····*·scrapy.utils.defer
131 ····*·scrapy.utils.log131 ····*·scrapy.utils.log
132 ····*·scrapy.utils.reactor132 ····*·scrapy.utils.reactor
133 ····*·scrapy.utils.request133 ····*·scrapy.utils.request
134 ····*·scrapy.utils.trackref134 ····*·scrapy.utils.trackref
  
135 ===============================================================================135 ===============================================================================
136 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.136 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
137 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.137 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/_modules/itemadapter/adapter.html
    
Offset 514, 15 lines modifiedOffset 514, 15 lines modified
514 ···········</div>514 ···········</div>
515 ··········</div>515 ··········</div>
516 ··········<footer>516 ··········<footer>
  
517 ··<hr/>517 ··<hr/>
  
518 ··<div·role="contentinfo">518 ··<div·role="contentinfo">
519 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.519 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
520 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.520 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
521 ······</span></p>521 ······</span></p>
522 ··</div>522 ··</div>
  
523 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a523 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
524 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>524 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
525 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.525 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
496 B
html2text {}
    
Offset 475, 9 lines modifiedOffset 475, 9 lines modified
475 ····if·isinstance(obj,·ItemAdapter):475 ····if·isinstance(obj,·ItemAdapter):
476 ········return·obj.asdict()476 ········return·obj.asdict()
477 ····if·ItemAdapter.is_item(obj):477 ····if·ItemAdapter.is_item(obj):
478 ········return·ItemAdapter(obj).asdict()478 ········return·ItemAdapter(obj).asdict()
479 ····return·obj479 ····return·obj
  
480 ===============================================================================480 ===============================================================================
481 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.481 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
482 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.482 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.12 KB
./usr/share/doc/python-scrapy-doc/html/_modules/itemadapter/utils.html
    
Offset 276, 15 lines modifiedOffset 276, 15 lines modified
276 ···········</div>276 ···········</div>
277 ··········</div>277 ··········</div>
278 ··········<footer>278 ··········<footer>
  
279 ··<hr/>279 ··<hr/>
  
280 ··<div·role="contentinfo">280 ··<div·role="contentinfo">
281 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.281 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
282 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.282 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
283 ······</span></p>283 ······</span></p>
284 ··</div>284 ··</div>
  
285 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a285 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
286 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>286 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
287 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.287 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
467 B
html2text {}
    
Offset 210, 9 lines modifiedOffset 210, 9 lines modified
210 ········stacklevel=2,210 ········stacklevel=2,
211 ····)211 ····)
212 ····from·itemadapter.adapter·import·ScrapyItemAdapter212 ····from·itemadapter.adapter·import·ScrapyItemAdapter
  
213 ····return·ScrapyItemAdapter.is_item(obj)213 ····return·ScrapyItemAdapter.is_item(obj)
  
214 ===============================================================================214 ===============================================================================
215 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.215 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
216 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.216 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/itemloaders.html
    
Offset 578, 15 lines modifiedOffset 578, 15 lines modified
578 ···········</div>578 ···········</div>
579 ··········</div>579 ··········</div>
580 ··········<footer>580 ··········<footer>
  
581 ··<hr/>581 ··<hr/>
  
582 ··<div·role="contentinfo">582 ··<div·role="contentinfo">
583 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.583 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
584 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.584 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
585 ······</span></p>585 ······</span></p>
586 ··</div>586 ··</div>
  
587 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a587 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
588 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>588 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
589 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.589 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
528 B
html2text {}
    
Offset 535, 9 lines modifiedOffset 535, 9 lines modified
  
535 ····def·_get_cssvalues(self,·csss,·**kw):535 ····def·_get_cssvalues(self,·csss,·**kw):
536 ········self._check_selector_method()536 ········self._check_selector_method()
537 ········csss·=·arg_to_iter(csss)537 ········csss·=·arg_to_iter(csss)
538 ········return·flatten(self.selector.css(css).getall()·for·css·in·csss)538 ········return·flatten(self.selector.css(css).getall()·for·css·in·csss)
  
539 ===============================================================================539 ===============================================================================
540 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.540 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
541 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.541 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/parsel/selector.html
    
Offset 785, 15 lines modifiedOffset 785, 15 lines modified
785 ···········</div>785 ···········</div>
786 ··········</div>786 ··········</div>
787 ··········<footer>787 ··········<footer>
  
788 ··<hr/>788 ··<hr/>
  
789 ··<div·role="contentinfo">789 ··<div·role="contentinfo">
790 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.790 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
791 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.791 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
792 ······</span></p>792 ······</span></p>
793 ··</div>793 ··</div>
  
794 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a794 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
795 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>795 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
796 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.796 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
522 B
html2text {}
    
Offset 739, 9 lines modifiedOffset 739, 9 lines modified
739 ····def·__str__(self)·->·str:739 ····def·__str__(self)·->·str:
740 ········data·=·repr(shorten(self.get(),·width=40))740 ········data·=·repr(shorten(self.get(),·width=40))
741 ········return·f"<{type(self).__name__}·xpath={self._expr!r}·data={data}>"741 ········return·f"<{type(self).__name__}·xpath={self._expr!r}·data={data}>"
  
742 ····__repr__·=·__str__742 ····__repr__·=·__str__
  
743 ===============================================================================743 ===============================================================================
744 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.744 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
745 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.745 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/parsel/xpathfuncs.html
    
Offset 198, 15 lines modifiedOffset 198, 15 lines modified
198 ···········</div>198 ···········</div>
199 ··········</div>199 ··········</div>
200 ··········<footer>200 ··········<footer>
  
201 ··<hr/>201 ··<hr/>
  
202 ··<div·role="contentinfo">202 ··<div·role="contentinfo">
203 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.203 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
204 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.204 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
205 ······</span></p>205 ······</span></p>
206 ··</div>206 ··</div>
  
207 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a207 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
208 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>208 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
209 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.209 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
507 B
html2text {}
    
Offset 125, 9 lines modifiedOffset 125, 9 lines modified
125 ····node_cls·=·replace_html5_whitespaces("·",·node_cls)125 ····node_cls·=·replace_html5_whitespaces("·",·node_cls)
126 ····for·cls·in·classes:126 ····for·cls·in·classes:
127 ········if·"·"·+·cls·+·"·"·not·in·node_cls:127 ········if·"·"·+·cls·+·"·"·not·in·node_cls:
128 ············return·False128 ············return·False
129 ····return·True129 ····return·True
  
130 ===============================================================================130 ===============================================================================
131 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.131 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
132 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.132 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/contracts/default.html
    
Offset 248, 15 lines modifiedOffset 248, 15 lines modified
248 ···········</div>248 ···········</div>
249 ··········</div>249 ··········</div>
250 ··········<footer>250 ··········<footer>
  
251 ··<hr/>251 ··<hr/>
  
252 ··<div·role="contentinfo">252 ··<div·role="contentinfo">
253 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.253 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
254 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.254 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
255 ······</span></p>255 ······</span></p>
256 ··</div>256 ··</div>
  
257 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a257 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
258 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>258 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
259 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.259 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
585 B
html2text {}
    
Offset 178, 9 lines modifiedOffset 178, 9 lines modified
178 ················missing·=·[arg·for·arg·in·self.args·if·arg·not·in·ItemAdapter178 ················missing·=·[arg·for·arg·in·self.args·if·arg·not·in·ItemAdapter
179 (x)]179 (x)]
180 ················if·missing:180 ················if·missing:
181 ····················missing_fields·=·",·".join(missing)181 ····················missing_fields·=·",·".join(missing)
182 ····················raise·ContractFail(f"Missing·fields:·{missing_fields}")182 ····················raise·ContractFail(f"Missing·fields:·{missing_fields}")
  
183 ===============================================================================183 ===============================================================================
184 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.184 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
185 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.185 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/contracts.html
    
Offset 318, 15 lines modifiedOffset 318, 15 lines modified
318 ···········</div>318 ···········</div>
319 ··········</div>319 ··········</div>
320 ··········<footer>320 ··········<footer>
  
321 ··<hr/>321 ··<hr/>
  
322 ··<div·role="contentinfo">322 ··<div·role="contentinfo">
323 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.323 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
324 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.324 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
325 ······</span></p>325 ······</span></p>
326 ··</div>326 ··</div>
  
327 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a327 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
328 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>328 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
329 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.329 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
527 B
html2text {}
    
Offset 247, 9 lines modifiedOffset 247, 9 lines modified
247 ············return·f"[{spider}]·{method.__name__}·({desc})"247 ············return·f"[{spider}]·{method.__name__}·({desc})"
  
248 ····name·=·f"{spider}_{method.__name__}"248 ····name·=·f"{spider}_{method.__name__}"
249 ····setattr(ContractTestCase,·name,·lambda·x:·x)249 ····setattr(ContractTestCase,·name,·lambda·x:·x)
250 ····return·ContractTestCase(name)250 ····return·ContractTestCase(name)
  
251 ===============================================================================251 ===============================================================================
252 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.252 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
253 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.253 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/core/scheduler.html
    
Offset 495, 15 lines modifiedOffset 495, 15 lines modified
495 ···········</div>495 ···········</div>
496 ··········</div>496 ··········</div>
497 ··········<footer>497 ··········<footer>
  
498 ··<hr/>498 ··<hr/>
  
499 ··<div·role="contentinfo">499 ··<div·role="contentinfo">
500 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.500 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
501 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.501 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
502 ······</span></p>502 ······</span></p>
503 ··</div>503 ··</div>
  
504 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a504 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
505 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>505 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
506 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.506 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
546 B
html2text {}
    
Offset 477, 9 lines modifiedOffset 477, 9 lines modified
477 ············return·json.load(f)477 ············return·json.load(f)
  
478 ····def·_write_dqs_state(self,·dqdir:·str,·state:·list)·->·None:478 ····def·_write_dqs_state(self,·dqdir:·str,·state:·list)·->·None:
479 ········with·Path(dqdir,·"active.json").open("w",·encoding="utf-8")·as·f:479 ········with·Path(dqdir,·"active.json").open("w",·encoding="utf-8")·as·f:
480 ············json.dump(state,·f)480 ············json.dump(state,·f)
  
481 ===============================================================================481 ===============================================================================
482 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.482 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
483 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.483 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.12 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/crawler.html
    
Offset 533, 15 lines modifiedOffset 533, 15 lines modified
533 ···········</div>533 ···········</div>
534 ··········</div>534 ··········</div>
535 ··········<footer>535 ··········<footer>
  
536 ··<hr/>536 ··<hr/>
  
537 ··<div·role="contentinfo">537 ··<div·role="contentinfo">
538 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.538 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
539 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.539 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
540 ······</span></p>540 ······</span></p>
541 ··</div>541 ··</div>
  
542 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a542 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
543 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>543 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
544 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.544 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
479 B
html2text {}
    
Offset 476, 9 lines modifiedOffset 476, 9 lines modified
  
476 ········try:476 ········try:
477 ············reactor.stop()477 ············reactor.stop()
478 ········except·RuntimeError:··#·raised·if·already·stopped·or·in·shutdown·stage478 ········except·RuntimeError:··#·raised·if·already·stopped·or·in·shutdown·stage
479 ············pass479 ············pass
  
480 ===============================================================================480 ===============================================================================
481 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.481 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
482 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.482 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.24 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/ajaxcrawl.html
    
Offset 233, 15 lines modifiedOffset 233, 15 lines modified
233 ···········</div>233 ···········</div>
234 ··········</div>234 ··········</div>
235 ··········<footer>235 ··········<footer>
  
236 ··<hr/>236 ··<hr/>
  
237 ··<div·role="contentinfo">237 ··<div·role="contentinfo">
238 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.238 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
240 ······</span></p>240 ······</span></p>
241 ··</div>241 ··</div>
  
242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
546 B
html2text {}
    
Offset 165, 9 lines modifiedOffset 165, 9 lines modified
  
165 ····text·=·html.remove_tags_with_content(text,·("script",·"noscript"))165 ····text·=·html.remove_tags_with_content(text,·("script",·"noscript"))
166 ····text·=·html.replace_entities(text)166 ····text·=·html.replace_entities(text)
167 ····text·=·html.remove_comments(text)167 ····text·=·html.remove_comments(text)
168 ····return·_ajax_crawlable_re.search(text)·is·not·None168 ····return·_ajax_crawlable_re.search(text)·is·not·None
  
169 ===============================================================================169 ===============================================================================
170 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.170 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
171 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.171 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.28 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/cookies.html
    
Offset 281, 15 lines modifiedOffset 281, 15 lines modified
281 ···········</div>281 ···········</div>
282 ··········</div>282 ··········</div>
283 ··········<footer>283 ··········<footer>
  
284 ··<hr/>284 ··<hr/>
  
285 ··<div·role="contentinfo">285 ··<div·role="contentinfo">
286 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.286 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
287 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.287 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
288 ······</span></p>288 ······</span></p>
289 ··</div>289 ··</div>
  
290 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a290 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
291 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>291 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
292 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.292 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
592 B
html2text {}
    
Offset 211, 9 lines modifiedOffset 211, 9 lines modified
211 ············cookies·=·request.cookies211 ············cookies·=·request.cookies
212 ········formatted·=·filter(None,·(self._format_cookie(c,·request)·for·c·in212 ········formatted·=·filter(None,·(self._format_cookie(c,·request)·for·c·in
213 cookies))213 cookies))
214 ········response·=·Response(request.url,·headers={"Set-Cookie":·formatted})214 ········response·=·Response(request.url,·headers={"Set-Cookie":·formatted})
215 ········return·jar.make_cookies(response,·request)215 ········return·jar.make_cookies(response,·request)
  
216 ===============================================================================216 ===============================================================================
217 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.217 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
218 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.218 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.21 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/defaultheaders.html
    
Offset 158, 15 lines modifiedOffset 158, 15 lines modified
158 ···········</div>158 ···········</div>
159 ··········</div>159 ··········</div>
160 ··········<footer>160 ··········<footer>
  
161 ··<hr/>161 ··<hr/>
  
162 ··<div·role="contentinfo">162 ··<div·role="contentinfo">
163 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.163 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
164 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.164 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
165 ······</span></p>165 ······</span></p>
166 ··</div>166 ··</div>
  
167 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a167 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
168 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>168 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
169 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.169 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
505 B
html2text {}
    
Offset 84, 9 lines modifiedOffset 84, 9 lines modified
84 ········return·cls(headers.items())84 ········return·cls(headers.items())
  
85 ····def·process_request(self,·request,·spider):85 ····def·process_request(self,·request,·spider):
86 ········for·k,·v·in·self._headers:86 ········for·k,·v·in·self._headers:
87 ············request.headers.setdefault(k,·v)87 ············request.headers.setdefault(k,·v)
  
88 ===============================================================================88 ===============================================================================
89 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.89 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
90 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.90 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.26 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/downloadtimeout.html
    
Offset 162, 15 lines modifiedOffset 162, 15 lines modified
162 ···········</div>162 ···········</div>
163 ··········</div>163 ··········</div>
164 ··········<footer>164 ··········<footer>
  
165 ··<hr/>165 ··<hr/>
  
166 ··<div·role="contentinfo">166 ··<div·role="contentinfo">
167 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.167 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
168 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.168 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
169 ······</span></p>169 ······</span></p>
170 ··</div>170 ··</div>
  
171 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a171 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
172 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>172 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
173 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.173 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
561 B
html2text {}
    
Offset 87, 9 lines modifiedOffset 87, 9 lines modified
87 ········self._timeout·=·getattr(spider,·"download_timeout",·self._timeout)87 ········self._timeout·=·getattr(spider,·"download_timeout",·self._timeout)
  
88 ····def·process_request(self,·request,·spider):88 ····def·process_request(self,·request,·spider):
89 ········if·self._timeout:89 ········if·self._timeout:
90 ············request.meta.setdefault("download_timeout",·self._timeout)90 ············request.meta.setdefault("download_timeout",·self._timeout)
  
91 ===============================================================================91 ===============================================================================
92 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.92 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
93 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.93 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/httpauth.html
    
Offset 189, 15 lines modifiedOffset 189, 15 lines modified
189 ···········</div>189 ···········</div>
190 ··········</div>190 ··········</div>
191 ··········<footer>191 ··········<footer>
  
192 ··<hr/>192 ··<hr/>
  
193 ··<div·role="contentinfo">193 ··<div·role="contentinfo">
194 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.194 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
195 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.195 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
196 ······</span></p>196 ······</span></p>
197 ··</div>197 ··</div>
  
198 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a198 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
199 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>199 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
200 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.200 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
564 B
html2text {}
    
Offset 118, 9 lines modifiedOffset 118, 9 lines modified
118 ················self.domain·=·domain118 ················self.domain·=·domain
119 ················self.domain_unset·=·False119 ················self.domain_unset·=·False
120 ············if·not·self.domain·or·url_is_from_any_domain(request.url,120 ············if·not·self.domain·or·url_is_from_any_domain(request.url,
121 [self.domain]):121 [self.domain]):
122 ················request.headers[b"Authorization"]·=·auth122 ················request.headers[b"Authorization"]·=·auth
  
123 ===============================================================================123 ===============================================================================
124 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.124 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
125 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.125 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.32 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/httpcache.html
    
Offset 287, 15 lines modifiedOffset 287, 15 lines modified
287 ···········</div>287 ···········</div>
288 ··········</div>288 ··········</div>
289 ··········<footer>289 ··········<footer>
  
290 ··<hr/>290 ··<hr/>
  
291 ··<div·role="contentinfo">291 ··<div·role="contentinfo">
292 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.292 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
294 ······</span></p>294 ······</span></p>
295 ··</div>295 ··</div>
  
296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
628 B
html2text {}
    
Offset 215, 9 lines modifiedOffset 215, 9 lines modified
215 ········if·self.policy.should_cache_response(response,·request):215 ········if·self.policy.should_cache_response(response,·request):
216 ············self.stats.inc_value("httpcache/store",·spider=spider)216 ············self.stats.inc_value("httpcache/store",·spider=spider)
217 ············self.storage.store_response(spider,·request,·response)217 ············self.storage.store_response(spider,·request,·response)
218 ········else:218 ········else:
219 ············self.stats.inc_value("httpcache/uncacheable",·spider=spider)219 ············self.stats.inc_value("httpcache/uncacheable",·spider=spider)
  
220 ===============================================================================220 ===============================================================================
221 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.221 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
222 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.222 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/httpcompression.html
    
Offset 245, 15 lines modifiedOffset 245, 15 lines modified
245 ···········</div>245 ···········</div>
246 ··········</div>246 ··········</div>
247 ··········<footer>247 ··········<footer>
  
248 ··<hr/>248 ··<hr/>
  
249 ··<div·role="contentinfo">249 ··<div·role="contentinfo">
250 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.250 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
251 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.251 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
252 ······</span></p>252 ······</span></p>
253 ··</div>253 ··</div>
  
254 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a254 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
255 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>255 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
256 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.256 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
547 B
html2text {}
    
Offset 175, 9 lines modifiedOffset 175, 9 lines modified
175 ············#·where·there·is·content·size·data·embedded·in·the·frame175 ············#·where·there·is·content·size·data·embedded·in·the·frame
176 ············reader·=·zstandard.ZstdDecompressor().stream_reader(io.BytesIO176 ············reader·=·zstandard.ZstdDecompressor().stream_reader(io.BytesIO
177 (body))177 (body))
178 ············body·=·reader.read()178 ············body·=·reader.read()
179 ········return·body179 ········return·body
  
180 ===============================================================================180 ===============================================================================
181 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.181 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
182 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.182 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/httpproxy.html
    
Offset 217, 15 lines modifiedOffset 217, 15 lines modified
217 ···········</div>217 ···········</div>
218 ··········</div>218 ··········</div>
219 ··········<footer>219 ··········<footer>
  
220 ··<hr/>220 ··<hr/>
  
221 ··<div·role="contentinfo">221 ··<div·role="contentinfo">
222 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.222 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
223 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.223 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
224 ······</span></p>224 ······</span></p>
225 ··</div>225 ··</div>
  
226 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a226 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
227 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>227 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
228 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.228 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
558 B
html2text {}
    
Offset 144, 9 lines modifiedOffset 144, 9 lines modified
144 ········elif·b"Proxy-Authorization"·in·request.headers:144 ········elif·b"Proxy-Authorization"·in·request.headers:
145 ············if·proxy_url:145 ············if·proxy_url:
146 ················request.meta["_auth_proxy"]·=·proxy_url146 ················request.meta["_auth_proxy"]·=·proxy_url
147 ············else:147 ············else:
148 ················del·request.headers[b"Proxy-Authorization"]148 ················del·request.headers[b"Proxy-Authorization"]
  
149 ===============================================================================149 ===============================================================================
150 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.150 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
151 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.151 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/redirect.html
    
Offset 276, 15 lines modifiedOffset 276, 15 lines modified
276 ···········</div>276 ···········</div>
277 ··········</div>277 ··········</div>
278 ··········<footer>278 ··········<footer>
  
279 ··<hr/>279 ··<hr/>
  
280 ··<div·role="contentinfo">280 ··<div·role="contentinfo">
281 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.281 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
282 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.282 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
283 ······</span></p>283 ······</span></p>
284 ··</div>284 ··</div>
  
285 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a285 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
286 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>286 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
287 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.287 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
564 B
html2text {}
    
Offset 207, 9 lines modifiedOffset 207, 9 lines modified
207 ········if·url·and·interval·<·self._maxdelay:207 ········if·url·and·interval·<·self._maxdelay:
208 ············redirected·=·self._redirect_request_using_get(request,·url)208 ············redirected·=·self._redirect_request_using_get(request,·url)
209 ············return·self._redirect(redirected,·request,·spider,·"meta·refresh")209 ············return·self._redirect(redirected,·request,·spider,·"meta·refresh")
  
210 ········return·response210 ········return·response
  
211 ===============================================================================211 ===============================================================================
212 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.212 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
213 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.213 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.18 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/retry.html
    
Offset 315, 15 lines modifiedOffset 315, 15 lines modified
315 ···········</div>315 ···········</div>
316 ··········</div>316 ··········</div>
317 ··········<footer>317 ··········<footer>
  
318 ··<hr/>318 ··<hr/>
  
319 ··<div·role="contentinfo">319 ··<div·role="contentinfo">
320 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.320 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
321 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.321 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
322 ······</span></p>322 ······</span></p>
323 ··</div>323 ··</div>
  
324 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a324 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
325 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>325 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
326 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.326 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
496 B
html2text {}
    
Offset 246, 9 lines modifiedOffset 246, 9 lines modified
246 ············reason=reason,246 ············reason=reason,
247 ············spider=spider,247 ············spider=spider,
248 ············max_retry_times=max_retry_times,248 ············max_retry_times=max_retry_times,
249 ············priority_adjust=priority_adjust,249 ············priority_adjust=priority_adjust,
250 ········)250 ········)
  
251 ===============================================================================251 ===============================================================================
252 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.252 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
253 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.253 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/robotstxt.html
    
Offset 258, 15 lines modifiedOffset 258, 15 lines modified
258 ···········</div>258 ···········</div>
259 ··········</div>259 ··········</div>
260 ··········<footer>260 ··········<footer>
  
261 ··<hr/>261 ··<hr/>
  
262 ··<div·role="contentinfo">262 ··<div·role="contentinfo">
263 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.263 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
264 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.264 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
265 ······</span></p>265 ······</span></p>
266 ··</div>266 ··</div>
  
267 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a267 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
268 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>268 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
269 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.269 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
556 B
html2text {}
    
Offset 187, 9 lines modifiedOffset 187, 9 lines modified
187 ············key·=·f"robotstxt/exception_count/{failure.type}"187 ············key·=·f"robotstxt/exception_count/{failure.type}"
188 ············self.crawler.stats.inc_value(key)188 ············self.crawler.stats.inc_value(key)
189 ········rp_dfd·=·self._parsers[netloc]189 ········rp_dfd·=·self._parsers[netloc]
190 ········self._parsers[netloc]·=·None190 ········self._parsers[netloc]·=·None
191 ········rp_dfd.callback(None)191 ········rp_dfd.callback(None)
  
192 ===============================================================================192 ===============================================================================
193 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.193 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
194 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.194 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.27 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/stats.html
    
Offset 197, 15 lines modifiedOffset 197, 15 lines modified
197 ···········</div>197 ···········</div>
198 ··········</div>198 ··········</div>
199 ··········<footer>199 ··········<footer>
  
200 ··<hr/>200 ··<hr/>
  
201 ··<div·role="contentinfo">201 ··<div·role="contentinfo">
202 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.202 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
203 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.203 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
204 ······</span></p>204 ······</span></p>
205 ··</div>205 ··</div>
  
206 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a206 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
207 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>207 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
208 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.208 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
588 B
html2text {}
    
Offset 124, 9 lines modifiedOffset 124, 9 lines modified
124 ········ex_class·=·global_object_name(exception.__class__)124 ········ex_class·=·global_object_name(exception.__class__)
125 ········self.stats.inc_value("downloader/exception_count",·spider=spider)125 ········self.stats.inc_value("downloader/exception_count",·spider=spider)
126 ········self.stats.inc_value(126 ········self.stats.inc_value(
127 ············f"downloader/exception_type_count/{ex_class}",·spider=spider127 ············f"downloader/exception_type_count/{ex_class}",·spider=spider
128 ········)128 ········)
  
129 ===============================================================================129 ===============================================================================
130 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.130 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
131 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.131 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/downloadermiddlewares/useragent.html
    
Offset 160, 15 lines modifiedOffset 160, 15 lines modified
160 ···········</div>160 ···········</div>
161 ··········</div>161 ··········</div>
162 ··········<footer>162 ··········<footer>
  
163 ··<hr/>163 ··<hr/>
  
164 ··<div·role="contentinfo">164 ··<div·role="contentinfo">
165 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.165 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
166 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.166 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
167 ······</span></p>167 ······</span></p>
168 ··</div>168 ··</div>
  
169 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a169 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
170 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>170 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
171 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.171 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
561 B
html2text {}
    
Offset 85, 9 lines modifiedOffset 85, 9 lines modified
85 ········self.user_agent·=·getattr(spider,·"user_agent",·self.user_agent)85 ········self.user_agent·=·getattr(spider,·"user_agent",·self.user_agent)
  
86 ····def·process_request(self,·request,·spider):86 ····def·process_request(self,·request,·spider):
87 ········if·self.user_agent:87 ········if·self.user_agent:
88 ············request.headers.setdefault(b"User-Agent",·self.user_agent)88 ············request.headers.setdefault(b"User-Agent",·self.user_agent)
  
89 ===============================================================================89 ===============================================================================
90 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.90 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
91 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.91 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.1 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/exceptions.html
    
Offset 233, 15 lines modifiedOffset 233, 15 lines modified
233 ···········</div>233 ···········</div>
234 ··········</div>234 ··········</div>
235 ··········<footer>235 ··········<footer>
  
236 ··<hr/>236 ··<hr/>
  
237 ··<div·role="contentinfo">237 ··<div·role="contentinfo">
238 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.238 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
240 ······</span></p>240 ······</span></p>
241 ··</div>241 ··</div>
  
242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
448 B
html2text {}
    
Offset 168, 9 lines modifiedOffset 168, 9 lines modified
  
168 [docs]class·ContractFail(AssertionError):168 [docs]class·ContractFail(AssertionError):
169 ····"""Error·raised·in·case·of·a·failing·contract"""169 ····"""Error·raised·in·case·of·a·failing·contract"""
  
170 ····pass170 ····pass
  
171 ===============================================================================171 ===============================================================================
172 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.172 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
173 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.173 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.18 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/exporters.html
    
Offset 499, 15 lines modifiedOffset 499, 15 lines modified
499 ···········</div>499 ···········</div>
500 ··········</div>500 ··········</div>
501 ··········<footer>501 ··········<footer>
  
502 ··<hr/>502 ··<hr/>
  
503 ··<div·role="contentinfo">503 ··<div·role="contentinfo">
504 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.504 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
505 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.505 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
506 ······</span></p>506 ······</span></p>
507 ··</div>507 ··</div>
  
508 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a508 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
509 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>509 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
510 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.510 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
534 B
html2text {}
    
Offset 446, 9 lines modifiedOffset 446, 9 lines modified
446 ····def·export_item(self,·item):446 ····def·export_item(self,·item):
447 ········result·=·dict(self._get_serialized_fields(item))447 ········result·=·dict(self._get_serialized_fields(item))
448 ········if·self.binary:448 ········if·self.binary:
449 ············result·=·dict(self._serialize_item(result))449 ············result·=·dict(self._serialize_item(result))
450 ········return·result450 ········return·result
  
451 ===============================================================================451 ===============================================================================
452 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.452 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
453 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.453 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/closespider.html
    
Offset 208, 15 lines modifiedOffset 208, 15 lines modified
208 ···········</div>208 ···········</div>
209 ··········</div>209 ··········</div>
210 ··········<footer>210 ··········<footer>
  
211 ··<hr/>211 ··<hr/>
  
212 ··<div·role="contentinfo">212 ··<div·role="contentinfo">
213 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.213 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
214 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.214 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
215 ······</span></p>215 ······</span></p>
216 ··</div>216 ··</div>
  
217 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a217 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
218 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>218 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
219 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.219 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
485 B
html2text {}
    
Offset 138, 9 lines modifiedOffset 138, 9 lines modified
  
138 ····def·spider_closed(self,·spider):138 ····def·spider_closed(self,·spider):
139 ········task·=·getattr(self,·"task",·False)139 ········task·=·getattr(self,·"task",·False)
140 ········if·task·and·task.active():140 ········if·task·and·task.active():
141 ············task.cancel()141 ············task.cancel()
  
142 ===============================================================================142 ===============================================================================
143 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.143 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
144 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.144 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.26 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/corestats.html
    
Offset 184, 15 lines modifiedOffset 184, 15 lines modified
184 ···········</div>184 ···········</div>
185 ··········</div>185 ··········</div>
186 ··········<footer>186 ··········<footer>
  
187 ··<hr/>187 ··<hr/>
  
188 ··<div·role="contentinfo">188 ··<div·role="contentinfo">
189 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.189 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
190 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.190 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
191 ······</span></p>191 ······</span></p>
192 ··</div>192 ··</div>
  
193 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a193 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
194 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>194 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
195 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.195 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
591 B
html2text {}
    
Offset 111, 9 lines modifiedOffset 111, 9 lines modified
111 ····def·item_dropped(self,·item,·spider,·exception):111 ····def·item_dropped(self,·item,·spider,·exception):
112 ········reason·=·exception.__class__.__name__112 ········reason·=·exception.__class__.__name__
113 ········self.stats.inc_value("item_dropped_count",·spider=spider)113 ········self.stats.inc_value("item_dropped_count",·spider=spider)
114 ········self.stats.inc_value(f"item_dropped_reasons_count/{reason}",114 ········self.stats.inc_value(f"item_dropped_reasons_count/{reason}",
115 spider=spider)115 spider=spider)
  
116 ===============================================================================116 ===============================================================================
117 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.117 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
118 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.118 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/debug.html
    
Offset 203, 15 lines modifiedOffset 203, 15 lines modified
203 ···········</div>203 ···········</div>
204 ··········</div>204 ··········</div>
205 ··········<footer>205 ··········<footer>
  
206 ··<hr/>206 ··<hr/>
  
207 ··<div·role="contentinfo">207 ··<div·role="contentinfo">
208 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.208 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
209 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.209 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
210 ······</span></p>210 ······</span></p>
211 ··</div>211 ··</div>
  
212 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a212 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
213 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>213 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
214 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.214 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
503 B
html2text {}
    
Offset 129, 9 lines modifiedOffset 129, 9 lines modified
129 ············#·win32·platforms·don't·support·SIGUSR·signals129 ············#·win32·platforms·don't·support·SIGUSR·signals
130 ············pass130 ············pass
  
131 ····def·_enter_debugger(self,·signum,·frame):131 ····def·_enter_debugger(self,·signum,·frame):
132 ········Pdb().set_trace(frame.f_back)132 ········Pdb().set_trace(frame.f_back)
  
133 ===============================================================================133 ===============================================================================
134 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.134 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
135 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.135 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.26 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/feedexport.html
    
Offset 741, 15 lines modifiedOffset 741, 15 lines modified
741 ···········</div>741 ···········</div>
742 ··········</div>742 ··········</div>
743 ··········<footer>743 ··········<footer>
  
744 ··<hr/>744 ··<hr/>
  
745 ··<div·role="contentinfo">745 ··<div·role="contentinfo">
746 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.746 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
747 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.747 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
748 ······</span></p>748 ······</span></p>
749 ··</div>749 ··</div>
  
750 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a750 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
751 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>751 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
752 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.752 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
594 B
html2text {}
    
Offset 698, 9 lines modifiedOffset 698, 9 lines modified
698 ····def·_load_filter(self,·feed_options):698 ····def·_load_filter(self,·feed_options):
699 ········#·load·the·item·filter·if·declared·else·load·the·default·filter·class699 ········#·load·the·item·filter·if·declared·else·load·the·default·filter·class
700 ········item_filter_class·=·load_object(feed_options.get("item_filter",700 ········item_filter_class·=·load_object(feed_options.get("item_filter",
701 ItemFilter))701 ItemFilter))
702 ········return·item_filter_class(feed_options)702 ········return·item_filter_class(feed_options)
  
703 ===============================================================================703 ===============================================================================
704 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.704 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
705 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.705 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/httpcache.html
    
Offset 520, 15 lines modifiedOffset 520, 15 lines modified
520 ···········</div>520 ···········</div>
521 ··········</div>521 ··········</div>
522 ··········<footer>522 ··········<footer>
  
523 ··<hr/>523 ··<hr/>
  
524 ··<div·role="contentinfo">524 ··<div·role="contentinfo">
525 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.525 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
526 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.526 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
527 ······</span></p>527 ······</span></p>
528 ··</div>528 ··</div>
  
529 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a529 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
530 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>530 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
531 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.531 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
500 B
html2text {}
    
Offset 456, 9 lines modifiedOffset 456, 9 lines modified
456 ····try:456 ····try:
457 ········date_str·=·to_unicode(date_str,·encoding="ascii")457 ········date_str·=·to_unicode(date_str,·encoding="ascii")
458 ········return·mktime_tz(parsedate_tz(date_str))458 ········return·mktime_tz(parsedate_tz(date_str))
459 ····except·Exception:459 ····except·Exception:
460 ········return·None460 ········return·None
  
461 ===============================================================================461 ===============================================================================
462 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.462 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
463 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.463 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/logstats.html
    
Offset 195, 15 lines modifiedOffset 195, 15 lines modified
195 ···········</div>195 ···········</div>
196 ··········</div>196 ··········</div>
197 ··········<footer>197 ··········<footer>
  
198 ··<hr/>198 ··<hr/>
  
199 ··<div·role="contentinfo">199 ··<div·role="contentinfo">
200 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.200 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
201 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.201 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
202 ······</span></p>202 ······</span></p>
203 ··</div>203 ··</div>
  
204 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a204 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
205 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>205 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
206 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.206 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
522 B
html2text {}
    
Offset 120, 9 lines modifiedOffset 120, 9 lines modified
120 ········logger.info(msg,·log_args,·extra={"spider":·spider})120 ········logger.info(msg,·log_args,·extra={"spider":·spider})
  
121 ····def·spider_closed(self,·spider,·reason):121 ····def·spider_closed(self,·spider,·reason):
122 ········if·self.task·and·self.task.running:122 ········if·self.task·and·self.task.running:
123 ············self.task.stop()123 ············self.task.stop()
  
124 ===============================================================================124 ===============================================================================
125 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.125 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
126 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.126 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/memdebug.html
    
Offset 173, 15 lines modifiedOffset 173, 15 lines modified
173 ···········</div>173 ···········</div>
174 ··········</div>174 ··········</div>
175 ··········<footer>175 ··········<footer>
  
176 ··<hr/>176 ··<hr/>
  
177 ··<div·role="contentinfo">177 ··<div·role="contentinfo">
178 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.178 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
179 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.179 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
180 ······</span></p>180 ······</span></p>
181 ··</div>181 ··</div>
  
182 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a182 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
183 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>183 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
184 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.184 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
519 B
html2text {}
    
Offset 98, 9 lines modifiedOffset 98, 9 lines modified
98 ············if·not·wdict:98 ············if·not·wdict:
99 ················continue99 ················continue
100 ············self.stats.set_value(100 ············self.stats.set_value(
101 ················f"memdebug/live_refs/{cls.__name__}",·len(wdict),·spider=spider101 ················f"memdebug/live_refs/{cls.__name__}",·len(wdict),·spider=spider
102 ············)102 ············)
  
103 ===============================================================================103 ===============================================================================
104 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.104 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
105 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.105 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/memusage.html
    
Offset 278, 15 lines modifiedOffset 278, 15 lines modified
278 ···········</div>278 ···········</div>
279 ··········</div>279 ··········</div>
280 ··········<footer>280 ··········<footer>
  
281 ··<hr/>281 ··<hr/>
  
282 ··<div·role="contentinfo">282 ··<div·role="contentinfo">
283 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.283 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
284 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.284 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
285 ······</span></p>285 ······</span></p>
286 ··</div>286 ··</div>
  
287 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a287 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
288 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>288 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
289 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.289 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
495 B
html2text {}
    
Offset 212, 9 lines modifiedOffset 212, 9 lines modified
212 ········)212 ········)
213 ········s·+=·"\r\n"213 ········s·+=·"\r\n"
214 ········s·+=·pformat(get_engine_status(self.crawler.engine))214 ········s·+=·pformat(get_engine_status(self.crawler.engine))
215 ········s·+=·"\r\n"215 ········s·+=·"\r\n"
216 ········self.mail.send(rcpts,·subject,·s)216 ········self.mail.send(rcpts,·subject,·s)
  
217 ===============================================================================217 ===============================================================================
218 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.218 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
219 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.219 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/postprocessing.html
    
Offset 306, 15 lines modifiedOffset 306, 15 lines modified
306 ···········</div>306 ···········</div>
307 ··········</div>307 ··········</div>
308 ··········<footer>308 ··········<footer>
  
309 ··<hr/>309 ··<hr/>
  
310 ··<div·role="contentinfo">310 ··<div·role="contentinfo">
311 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.311 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
312 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.312 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
313 ······</span></p>313 ······</span></p>
314 ··</div>314 ··</div>
  
315 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a315 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
316 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>316 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
317 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.317 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
519 B
html2text {}
    
Offset 239, 9 lines modifiedOffset 239, 9 lines modified
239 ····def·_get_head_plugin(self)·->·Any:239 ····def·_get_head_plugin(self)·->·Any:
240 ········prev·=·self.file240 ········prev·=·self.file
241 ········for·plugin·in·self.plugins[::-1]:241 ········for·plugin·in·self.plugins[::-1]:
242 ············prev·=·plugin(prev,·self.feed_options)242 ············prev·=·plugin(prev,·self.feed_options)
243 ········return·prev243 ········return·prev
  
244 ===============================================================================244 ===============================================================================
245 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.245 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
246 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.246 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.24 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/statsmailer.html
    
Offset 171, 15 lines modifiedOffset 171, 15 lines modified
171 ···········</div>171 ···········</div>
172 ··········</div>172 ··········</div>
173 ··········<footer>173 ··········<footer>
  
174 ··<hr/>174 ··<hr/>
  
175 ··<div·role="contentinfo">175 ··<div·role="contentinfo">
176 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.176 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
177 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.177 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
178 ······</span></p>178 ······</span></p>
179 ··</div>179 ··</div>
  
180 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a180 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
181 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>181 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
182 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.182 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
565 B
html2text {}
    
Offset 98, 9 lines modifiedOffset 98, 9 lines modified
98 ().items())98 ().items())
99 ········body·+=·f"\n\n{spider.name}·stats\n\n"99 ········body·+=·f"\n\n{spider.name}·stats\n\n"
100 ········body·+=·"\n".join(f"{k:<50}·:·{v}"·for·k,·v·in·spider_stats.items())100 ········body·+=·"\n".join(f"{k:<50}·:·{v}"·for·k,·v·in·spider_stats.items())
101 ········return·self.mail.send(self.recipients,·f"Scrapy·stats·for:101 ········return·self.mail.send(self.recipients,·f"Scrapy·stats·for:
102 {spider.name}",·body)102 {spider.name}",·body)
  
103 ===============================================================================103 ===============================================================================
104 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.104 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
105 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.105 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.21 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/extensions/telnet.html
    
Offset 252, 15 lines modifiedOffset 252, 15 lines modified
252 ···········</div>252 ···········</div>
253 ··········</div>253 ··········</div>
254 ··········<footer>254 ··········<footer>
  
255 ··<hr/>255 ··<hr/>
  
256 ··<div·role="contentinfo">256 ··<div·role="contentinfo">
257 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.257 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
258 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.258 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
259 ······</span></p>259 ······</span></p>
260 ··</div>260 ··</div>
  
261 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a261 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
262 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>262 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
263 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.263 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
543 B
html2text {}
    
Offset 182, 9 lines modifiedOffset 182, 9 lines modified
182 ············"https://docs.scrapy.org/en/latest/topics/telnetconsole.html",182 ············"https://docs.scrapy.org/en/latest/topics/telnetconsole.html",
183 ········}183 ········}
184 ········self.crawler.signals.send_catch_log(update_telnet_vars,184 ········self.crawler.signals.send_catch_log(update_telnet_vars,
185 telnet_vars=telnet_vars)185 telnet_vars=telnet_vars)
186 ········return·telnet_vars186 ········return·telnet_vars
  
187 ===============================================================================187 ===============================================================================
188 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.188 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
189 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.189 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/request/json_request.html
    
Offset 202, 15 lines modifiedOffset 202, 15 lines modified
202 ···········</div>202 ···········</div>
203 ··········</div>203 ··········</div>
204 ··········<footer>204 ··········<footer>
  
205 ··<hr/>205 ··<hr/>
  
206 ··<div·role="contentinfo">206 ··<div·role="contentinfo">
207 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.207 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
208 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.208 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
209 ······</span></p>209 ······</span></p>
210 ··</div>210 ··</div>
  
211 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a211 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
212 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>212 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
213 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.213 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
465 B
html2text {}
    
Offset 130, 9 lines modifiedOffset 130, 9 lines modified
130 ········return·json.dumps(data,·**self._dumps_kwargs)130 ········return·json.dumps(data,·**self._dumps_kwargs)
  
  
  
131 JSONRequest·=·create_deprecated_class("JSONRequest",·JsonRequest)131 JSONRequest·=·create_deprecated_class("JSONRequest",·JsonRequest)
  
132 ===============================================================================132 ===============================================================================
133 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.133 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
134 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.134 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.27 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/request.html
    
Offset 385, 15 lines modifiedOffset 385, 15 lines modified
385 ···········</div>385 ···········</div>
386 ··········</div>386 ··········</div>
387 ··········<footer>387 ··········<footer>
  
388 ··<hr/>388 ··<hr/>
  
389 ··<div·role="contentinfo">389 ··<div·role="contentinfo">
390 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.390 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
391 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.391 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
392 ······</span></p>392 ······</span></p>
393 ··</div>393 ··</div>
  
394 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a394 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
395 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>395 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
396 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.396 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
616 B
html2text {}
    
Offset 330, 9 lines modifiedOffset 330, 9 lines modified
330 ············#·Reference:·The·standard·type·hierarchy330 ············#·Reference:·The·standard·type·hierarchy
331 ············#·https://docs.python.org/3/reference/datamodel.html331 ············#·https://docs.python.org/3/reference/datamodel.html
332 ············if·obj_func.__func__·is·func.__func__:332 ············if·obj_func.__func__·is·func.__func__:
333 ················return·name333 ················return·name
334 ····raise·ValueError(f"Function·{func}·is·not·an·instance·method·in:·{obj}")334 ····raise·ValueError(f"Function·{func}·is·not·an·instance·method·in:·{obj}")
  
335 ===============================================================================335 ===============================================================================
336 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.336 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
337 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.337 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.11 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/response/html.html
    
Offset 150, 15 lines modifiedOffset 150, 15 lines modified
150 ···········</div>150 ···········</div>
151 ··········</div>151 ··········</div>
152 ··········<footer>152 ··········<footer>
  
153 ··<hr/>153 ··<hr/>
  
154 ··<div·role="contentinfo">154 ··<div·role="contentinfo">
155 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.155 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
156 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.156 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
157 ······</span></p>157 ······</span></p>
158 ··</div>158 ··</div>
  
159 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a159 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
160 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>160 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
161 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.161 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
442 B
html2text {}
    
Offset 75, 9 lines modifiedOffset 75, 9 lines modified
75 from·scrapy.http.response.text·import·TextResponse75 from·scrapy.http.response.text·import·TextResponse
  
  
76 [docs]class·HtmlResponse(TextResponse):76 [docs]class·HtmlResponse(TextResponse):
77 ····pass77 ····pass
  
78 ===============================================================================78 ===============================================================================
79 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.79 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
80 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.80 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/response/text.html
    
Offset 434, 15 lines modifiedOffset 434, 15 lines modified
434 ···········</div>434 ···········</div>
435 ··········</div>435 ··········</div>
436 ··········<footer>436 ··········<footer>
  
437 ··<hr/>437 ··<hr/>
  
438 ··<div·role="contentinfo">438 ··<div·role="contentinfo">
439 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.439 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
440 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.440 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
441 ······</span></p>441 ······</span></p>
442 ··</div>442 ··</div>
  
443 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a443 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
444 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>444 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
445 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.445 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
524 B
html2text {}
    
Offset 379, 9 lines modifiedOffset 379, 9 lines modified
379 ····href·=·sel.root.get("href")379 ····href·=·sel.root.get("href")
380 ····if·href·is·None:380 ····if·href·is·None:
381 ········raise·_InvalidSelector(f"<{sel.root.tag}>·element·has·no·href381 ········raise·_InvalidSelector(f"<{sel.root.tag}>·element·has·no·href
382 attribute:·{sel}")382 attribute:·{sel}")
383 ····return·strip_html5_whitespace(href)383 ····return·strip_html5_whitespace(href)
  
384 ===============================================================================384 ===============================================================================
385 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.385 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
386 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.386 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.11 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/response/xml.html
    
Offset 150, 15 lines modifiedOffset 150, 15 lines modified
150 ···········</div>150 ···········</div>
151 ··········</div>151 ··········</div>
152 ··········<footer>152 ··········<footer>
  
153 ··<hr/>153 ··<hr/>
  
154 ··<div·role="contentinfo">154 ··<div·role="contentinfo">
155 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.155 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
156 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.156 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
157 ······</span></p>157 ······</span></p>
158 ··</div>158 ··</div>
  
159 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a159 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
160 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>160 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
161 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.161 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
441 B
html2text {}
    
Offset 75, 9 lines modifiedOffset 75, 9 lines modified
75 from·scrapy.http.response.text·import·TextResponse75 from·scrapy.http.response.text·import·TextResponse
  
  
76 [docs]class·XmlResponse(TextResponse):76 [docs]class·XmlResponse(TextResponse):
77 ····pass77 ····pass
  
78 ===============================================================================78 ===============================================================================
79 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.79 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
80 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.80 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.12 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/http/response.html
    
Offset 386, 15 lines modifiedOffset 386, 15 lines modified
386 ···········</div>386 ···········</div>
387 ··········</div>387 ··········</div>
388 ··········<footer>388 ··········<footer>
  
389 ··<hr/>389 ··<hr/>
  
390 ··<div·role="contentinfo">390 ··<div·role="contentinfo">
391 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.391 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
392 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.392 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
393 ······</span></p>393 ······</span></p>
394 ··</div>394 ··</div>
  
395 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a395 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
396 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>396 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
397 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.397 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
460 B
html2text {}
    
Offset 319, 9 lines modifiedOffset 319, 9 lines modified
319 ················cb_kwargs=cb_kwargs,319 ················cb_kwargs=cb_kwargs,
320 ················flags=flags,320 ················flags=flags,
321 ············)321 ············)
322 ············for·url·in·urls322 ············for·url·in·urls
323 ········)323 ········)
  
324 ===============================================================================324 ===============================================================================
325 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.325 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
326 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.326 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.13 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/item.html
    
Offset 256, 15 lines modifiedOffset 256, 15 lines modified
256 ···········</div>256 ···········</div>
257 ··········</div>257 ··········</div>
258 ··········<footer>258 ··········<footer>
  
259 ··<hr/>259 ··<hr/>
  
260 ··<div·role="contentinfo">260 ··<div·role="contentinfo">
261 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.261 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
262 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.262 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
263 ······</span></p>263 ······</span></p>
264 ··</div>264 ··</div>
  
265 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a265 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
266 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>266 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
267 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.267 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
493 B
html2text {}
    
Offset 189, 9 lines modifiedOffset 189, 9 lines modified
189 ········return·self.__class__(self)189 ········return·self.__class__(self)
  
190 ····def·deepcopy(self):190 ····def·deepcopy(self):
191 ········"""Return·a·:func:`~copy.deepcopy`·of·this·item."""191 ········"""Return·a·:func:`~copy.deepcopy`·of·this·item."""
192 ········return·deepcopy(self)192 ········return·deepcopy(self)
  
193 ===============================================================================193 ===============================================================================
194 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.194 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
195 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.195 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/link.html
    
Offset 192, 15 lines modifiedOffset 192, 15 lines modified
192 ···········</div>192 ···········</div>
193 ··········</div>193 ··········</div>
194 ··········<footer>194 ··········<footer>
  
195 ··<hr/>195 ··<hr/>
  
196 ··<div·role="contentinfo">196 ··<div·role="contentinfo">
197 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.197 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
198 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.198 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
199 ······</span></p>199 ······</span></p>
200 ··</div>200 ··</div>
  
201 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a201 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
202 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>202 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
203 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.203 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
523 B
html2text {}
    
Offset 122, 9 lines modifiedOffset 122, 9 lines modified
122 ····def·__repr__(self):122 ····def·__repr__(self):
123 ········return·(123 ········return·(
124 ············f"Link(url={self.url!r},·text={self.text!r},·"124 ············f"Link(url={self.url!r},·text={self.text!r},·"
125 ············f"fragment={self.fragment!r},·nofollow={self.nofollow!r})"125 ············f"fragment={self.fragment!r},·nofollow={self.nofollow!r})"
126 ········)126 ········)
  
127 ===============================================================================127 ===============================================================================
128 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.128 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
129 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.129 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/linkextractors/lxmlhtml.html
    
Offset 382, 15 lines modifiedOffset 382, 15 lines modified
382 ···········</div>382 ···········</div>
383 ··········</div>383 ··········</div>
384 ··········<footer>384 ··········<footer>
  
385 ··<hr/>385 ··<hr/>
  
386 ··<div·role="contentinfo">386 ··<div·role="contentinfo">
387 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.387 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
388 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.388 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
389 ······</span></p>389 ······</span></p>
390 ··</div>390 ··</div>
  
391 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a391 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
392 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>392 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
393 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.393 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
515 B
html2text {}
    
Offset 319, 9 lines modifiedOffset 319, 9 lines modified
319 base_url)319 base_url)
320 ············all_links.extend(self._process_links(links))320 ············all_links.extend(self._process_links(links))
321 ········if·self.link_extractor.unique:321 ········if·self.link_extractor.unique:
322 ············return·unique_list(all_links)322 ············return·unique_list(all_links)
323 ········return·all_links323 ········return·all_links
  
324 ===============================================================================324 ===============================================================================
325 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.325 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
326 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.326 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/loader.html
    
Offset 228, 15 lines modifiedOffset 228, 15 lines modified
228 ···········</div>228 ···········</div>
229 ··········</div>229 ··········</div>
230 ··········<footer>230 ··········<footer>
  
231 ··<hr/>231 ··<hr/>
  
232 ··<div·role="contentinfo">232 ··<div·role="contentinfo">
233 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.233 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
234 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.234 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
235 ······</span></p>235 ······</span></p>
236 ··</div>236 ··</div>
  
237 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a237 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
238 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>238 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
239 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.239 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
532 B
html2text {}
    
Offset 163, 9 lines modifiedOffset 163, 9 lines modified
163 ············except·AttributeError:163 ············except·AttributeError:
164 ················selector·=·None164 ················selector·=·None
165 ········context.update(response=response)165 ········context.update(response=response)
166 ········super().__init__(item=item,·selector=selector,·parent=parent,166 ········super().__init__(item=item,·selector=selector,·parent=parent,
167 **context)167 **context)
  
168 ===============================================================================168 ===============================================================================
169 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.169 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
170 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.170 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.07 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/logformatter.html
    
Offset 284, 15 lines modifiedOffset 284, 15 lines modified
284 ···········</div>284 ···········</div>
285 ··········</div>285 ··········</div>
286 ··········<footer>286 ··········<footer>
  
287 ··<hr/>287 ··<hr/>
  
288 ··<div·role="contentinfo">288 ··<div·role="contentinfo">
289 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.289 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
290 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.290 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
291 ······</span></p>291 ······</span></p>
292 ··</div>292 ··</div>
  
293 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a293 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
294 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>294 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
295 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.295 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
418 B
html2text {}
    
Offset 229, 9 lines modifiedOffset 229, 9 lines modified
  
  
229 ····@classmethod229 ····@classmethod
230 ····def·from_crawler(cls,·crawler):230 ····def·from_crawler(cls,·crawler):
231 ········return·cls()231 ········return·cls()
  
232 ===============================================================================232 ===============================================================================
233 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.233 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
234 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.234 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.09 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/mail.html
    
Offset 345, 15 lines modifiedOffset 345, 15 lines modified
345 ···········</div>345 ···········</div>
346 ··········</div>346 ··········</div>
347 ··········<footer>347 ··········<footer>
  
348 ··<hr/>348 ··<hr/>
  
349 ··<div·role="contentinfo">349 ··<div·role="contentinfo">
350 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.350 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
351 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.351 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
352 ······</span></p>352 ······</span></p>
353 ··</div>353 ··</div>
  
354 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a354 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
355 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>355 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
356 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.356 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
451 B
html2text {}
    
Offset 274, 9 lines modifiedOffset 274, 9 lines modified
274 ············d,274 ············d,
275 ············**factory_keywords275 ············**factory_keywords
276 ········)276 ········)
277 ········factory.noisy·=·False277 ········factory.noisy·=·False
278 ········return·factory278 ········return·factory
  
279 ===============================================================================279 ===============================================================================
280 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.280 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
281 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.281 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.23 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/pipelines/files.html
    
Offset 682, 15 lines modifiedOffset 682, 15 lines modified
682 ···········</div>682 ···········</div>
683 ··········</div>683 ··········</div>
684 ··········<footer>684 ··········<footer>
  
685 ··<hr/>685 ··<hr/>
  
686 ··<div·role="contentinfo">686 ··<div·role="contentinfo">
687 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.687 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
688 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.688 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
689 ······</span></p>689 ······</span></p>
690 ··</div>690 ··</div>
  
691 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a691 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
692 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>692 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
693 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.693 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
571 B
html2text {}
    
Offset 628, 9 lines modifiedOffset 628, 9 lines modified
628 ············media_ext·=·""628 ············media_ext·=·""
629 ············media_type·=·mimetypes.guess_type(request.url)[0]629 ············media_type·=·mimetypes.guess_type(request.url)[0]
630 ············if·media_type:630 ············if·media_type:
631 ················media_ext·=·mimetypes.guess_extension(media_type)631 ················media_ext·=·mimetypes.guess_extension(media_type)
632 ········return·f"full/{media_guid}{media_ext}"632 ········return·f"full/{media_guid}{media_ext}"
  
633 ===============================================================================633 ===============================================================================
634 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.634 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
635 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.635 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.22 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/pipelines/images.html
    
Offset 368, 15 lines modifiedOffset 368, 15 lines modified
368 ···········</div>368 ···········</div>
369 ··········</div>369 ··········</div>
370 ··········<footer>370 ··········<footer>
  
371 ··<hr/>371 ··<hr/>
  
372 ··<div·role="contentinfo">372 ··<div·role="contentinfo">
373 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.373 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
374 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.374 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
375 ······</span></p>375 ······</span></p>
376 ··</div>376 ··</div>
  
377 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a377 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
378 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>378 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
379 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.379 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
556 B
html2text {}
    
Offset 307, 9 lines modifiedOffset 307, 9 lines modified
  
307 [docs]····def·thumb_path(self,·request,·thumb_id,·response=None,·info=None,·*,307 [docs]····def·thumb_path(self,·request,·thumb_id,·response=None,·info=None,·*,
308 item=None):308 item=None):
309 ········thumb_guid·=·hashlib.sha1(to_bytes(request.url)).hexdigest()309 ········thumb_guid·=·hashlib.sha1(to_bytes(request.url)).hexdigest()
310 ········return·f"thumbs/{thumb_id}/{thumb_guid}.jpg"310 ········return·f"thumbs/{thumb_id}/{thumb_guid}.jpg"
  
311 ===============================================================================311 ===============================================================================
312 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.312 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
313 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.313 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/robotstxt.html
    
Offset 272, 15 lines modifiedOffset 272, 15 lines modified
272 ···········</div>272 ···········</div>
273 ··········</div>273 ··········</div>
274 ··········<footer>274 ··········<footer>
  
275 ··<hr/>275 ··<hr/>
  
276 ··<div·role="contentinfo">276 ··<div·role="contentinfo">
277 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.277 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
278 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.278 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
279 ······</span></p>279 ······</span></p>
280 ··</div>280 ··</div>
  
281 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a281 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
282 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>282 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
283 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.283 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
507 B
html2text {}
    
Offset 204, 9 lines modifiedOffset 204, 9 lines modified
  
204 ····def·allowed(self,·url,·user_agent):204 ····def·allowed(self,·url,·user_agent):
205 ········user_agent·=·to_unicode(user_agent)205 ········user_agent·=·to_unicode(user_agent)
206 ········url·=·to_unicode(url)206 ········url·=·to_unicode(url)
207 ········return·self.rp.can_fetch(url,·user_agent)207 ········return·self.rp.can_fetch(url,·user_agent)
  
208 ===============================================================================208 ===============================================================================
209 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.209 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
210 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.210 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/selector/unified.html
    
Offset 220, 15 lines modifiedOffset 220, 15 lines modified
220 ···········</div>220 ···········</div>
221 ··········</div>221 ··········</div>
222 ··········<footer>222 ··········<footer>
  
223 ··<hr/>223 ··<hr/>
  
224 ··<div·role="contentinfo">224 ··<div·role="contentinfo">
225 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.225 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
226 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.226 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
227 ······</span></p>227 ······</span></p>
228 ··</div>228 ··</div>
  
229 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a229 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
230 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>230 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
231 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.231 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
531 B
html2text {}
    
Offset 149, 9 lines modifiedOffset 149, 9 lines modified
149 ············text·=·response.text149 ············text·=·response.text
150 ············kwargs.setdefault("base_url",·response.url)150 ············kwargs.setdefault("base_url",·response.url)
  
151 ········self.response·=·response151 ········self.response·=·response
152 ········super().__init__(text=text,·type=st,·root=root,·**kwargs)152 ········super().__init__(text=text,·type=st,·root=root,·**kwargs)
  
153 ===============================================================================153 ===============================================================================
154 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.154 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
155 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.155 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.23 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/settings.html
    
Offset 616, 15 lines modifiedOffset 616, 15 lines modified
616 ···········</div>616 ···········</div>
617 ··········</div>617 ··········</div>
618 ··········<footer>618 ··········<footer>
  
619 ··<hr/>619 ··<hr/>
  
620 ··<div·role="contentinfo">620 ··<div·role="contentinfo">
621 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.621 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
622 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.622 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
623 ······</span></p>623 ······</span></p>
624 ··</div>624 ··</div>
  
625 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a625 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
626 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>626 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
627 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.627 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
585 B
html2text {}
    
Offset 568, 9 lines modifiedOffset 568, 9 lines modified
568 ····"""Return·a·dict·of·the·settings·that·have·been·overridden"""568 ····"""Return·a·dict·of·the·settings·that·have·been·overridden"""
569 ····for·name,·defvalue·in·iter_default_settings():569 ····for·name,·defvalue·in·iter_default_settings():
570 ········value·=·settings[name]570 ········value·=·settings[name]
571 ········if·not·isinstance(defvalue,·dict)·and·value·!=·defvalue:571 ········if·not·isinstance(defvalue,·dict)·and·value·!=·defvalue:
572 ············yield·name,·value572 ············yield·name,·value
  
573 ===============================================================================573 ===============================================================================
574 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.574 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
575 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.575 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/signalmanager.html
    
Offset 205, 15 lines modifiedOffset 205, 15 lines modified
205 ···········</div>205 ···········</div>
206 ··········</div>206 ··········</div>
207 ··········<footer>207 ··········<footer>
  
208 ··<hr/>208 ··<hr/>
  
209 ··<div·role="contentinfo">209 ··<div·role="contentinfo">
210 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.210 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
212 ······</span></p>212 ······</span></p>
213 ··</div>213 ··</div>
  
214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
541 B
html2text {}
    
Offset 134, 9 lines modifiedOffset 134, 9 lines modified
134 ········:param·signal:·the·signal·to·disconnect·from134 ········:param·signal:·the·signal·to·disconnect·from
135 ········:type·signal:·object135 ········:type·signal:·object
136 ········"""136 ········"""
137 ········kwargs.setdefault("sender",·self.sender)137 ········kwargs.setdefault("sender",·self.sender)
138 ········return·_signal.disconnect_all(signal,·**kwargs)138 ········return·_signal.disconnect_all(signal,·**kwargs)
  
139 ===============================================================================139 ===============================================================================
140 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.140 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
141 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.141 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spiderloader.html
    
Offset 230, 15 lines modifiedOffset 230, 15 lines modified
230 ···········</div>230 ···········</div>
231 ··········</div>231 ··········</div>
232 ··········<footer>232 ··········<footer>
  
233 ··<hr/>233 ··<hr/>
  
234 ··<div·role="contentinfo">234 ··<div·role="contentinfo">
235 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.235 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
236 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.236 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
237 ······</span></p>237 ······</span></p>
238 ··</div>238 ··</div>
  
239 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a239 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
240 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>240 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
241 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.241 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
512 B
html2text {}
    
Offset 159, 9 lines modifiedOffset 159, 9 lines modified
159 [docs]····def·list(self):159 [docs]····def·list(self):
160 ········"""160 ········"""
161 ········Return·a·list·with·the·names·of·all·spiders·available·in·the·project.161 ········Return·a·list·with·the·names·of·all·spiders·available·in·the·project.
162 ········"""162 ········"""
163 ········return·list(self._spiders.keys())163 ········return·list(self._spiders.keys())
  
164 ===============================================================================164 ===============================================================================
165 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.165 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
166 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.166 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.24 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spidermiddlewares/depth.html
    
Offset 200, 15 lines modifiedOffset 200, 15 lines modified
200 ···········</div>200 ···········</div>
201 ··········</div>201 ··········</div>
202 ··········<footer>202 ··········<footer>
  
203 ··<hr/>203 ··<hr/>
  
204 ··<div·role="contentinfo">204 ··<div·role="contentinfo">
205 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.205 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
206 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.206 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
207 ······</span></p>207 ······</span></p>
208 ··</div>208 ··</div>
  
209 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a209 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
210 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>210 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
211 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.211 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
570 B
html2text {}
    
Offset 125, 9 lines modifiedOffset 125, 9 lines modified
125 ············return·False125 ············return·False
126 ········if·self.verbose_stats:126 ········if·self.verbose_stats:
127 ············self.stats.inc_value(f"request_depth_count/{depth}",·spider=spider)127 ············self.stats.inc_value(f"request_depth_count/{depth}",·spider=spider)
128 ········self.stats.max_value("request_depth_max",·depth,·spider=spider)128 ········self.stats.max_value("request_depth_max",·depth,·spider=spider)
129 ········return·True129 ········return·True
  
130 ===============================================================================130 ===============================================================================
131 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.131 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
132 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.132 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spidermiddlewares/httperror.html
    
Offset 196, 15 lines modifiedOffset 196, 15 lines modified
196 ···········</div>196 ···········</div>
197 ··········</div>197 ··········</div>
198 ··········<footer>198 ··········<footer>
  
199 ··<hr/>199 ··<hr/>
  
200 ··<div·role="contentinfo">200 ··<div·role="contentinfo">
201 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.201 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
202 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.202 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
203 ······</span></p>203 ······</span></p>
204 ··</div>204 ··</div>
  
205 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a205 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
206 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>206 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
207 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.207 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
485 B
html2text {}
    
Offset 123, 9 lines modifiedOffset 123, 9 lines modified
123 handled·or·not·allowed",123 handled·or·not·allowed",
124 ················{"response":·response},124 ················{"response":·response},
125 ················extra={"spider":·spider},125 ················extra={"spider":·spider},
126 ············)126 ············)
127 ············return·[]127 ············return·[]
  
128 ===============================================================================128 ===============================================================================
129 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.129 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
130 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.130 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.07 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spidermiddlewares/offsite.html
    
Offset 233, 15 lines modifiedOffset 233, 15 lines modified
233 ···········</div>233 ···········</div>
234 ··········</div>234 ··········</div>
235 ··········<footer>235 ··········<footer>
  
236 ··<hr/>236 ··<hr/>
  
237 ··<div·role="contentinfo">237 ··<div·role="contentinfo">
238 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.238 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.239 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
240 ······</span></p>240 ······</span></p>
241 ··</div>241 ··</div>
  
242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a242 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>243 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.244 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
390 B
html2text {}
    
Offset 159, 9 lines modifiedOffset 159, 9 lines modified
159 ····pass159 ····pass
  
  
160 class·PortWarning(Warning):160 class·PortWarning(Warning):
161 ····pass161 ····pass
  
162 ===============================================================================162 ===============================================================================
163 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.163 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
164 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.164 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.27 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spidermiddlewares/referer.html
    
Offset 523, 15 lines modifiedOffset 523, 15 lines modified
523 ···········</div>523 ···········</div>
524 ··········</div>524 ··········</div>
525 ··········<footer>525 ··········<footer>
  
526 ··<hr/>526 ··<hr/>
  
527 ··<div·role="contentinfo">527 ··<div·role="contentinfo">
528 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.528 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
529 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.529 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
530 ······</span></p>530 ······</span></p>
531 ··</div>531 ··</div>
  
532 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a532 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
533 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>533 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
534 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.534 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
596 B
html2text {}
    
Offset 479, 9 lines modifiedOffset 479, 9 lines modified
479 ················if·policy_referrer·!=·request_referrer:479 ················if·policy_referrer·!=·request_referrer:
480 ····················if·policy_referrer·is·None:480 ····················if·policy_referrer·is·None:
481 ························request.headers.pop("Referer")481 ························request.headers.pop("Referer")
482 ····················else:482 ····················else:
483 ························request.headers["Referer"]·=·policy_referrer483 ························request.headers["Referer"]·=·policy_referrer
  
484 ===============================================================================484 ===============================================================================
485 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.485 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
486 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.486 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.19 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spidermiddlewares/urllength.html
    
Offset 182, 15 lines modifiedOffset 182, 15 lines modified
182 ···········</div>182 ···········</div>
183 ··········</div>183 ··········</div>
184 ··········<footer>184 ··········<footer>
  
185 ··<hr/>185 ··<hr/>
  
186 ··<div·role="contentinfo">186 ··<div·role="contentinfo">
187 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.187 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
188 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.188 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
189 ······</span></p>189 ······</span></p>
190 ··</div>190 ··</div>
  
191 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a191 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
192 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>192 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
193 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.193 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
510 B
html2text {}
    
Offset 107, 9 lines modifiedOffset 107, 9 lines modified
107 ············spider.crawler.stats.inc_value(107 ············spider.crawler.stats.inc_value(
108 ················"urllength/request_ignored_count",·spider=spider108 ················"urllength/request_ignored_count",·spider=spider
109 ············)109 ············)
110 ············return·False110 ············return·False
111 ········return·True111 ········return·True
  
112 ===============================================================================112 ===============================================================================
113 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.113 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
114 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.114 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spiders/crawl.html
    
Offset 287, 15 lines modifiedOffset 287, 15 lines modified
287 ···········</div>287 ···········</div>
288 ··········</div>288 ··········</div>
289 ··········<footer>289 ··········<footer>
  
290 ··<hr/>290 ··<hr/>
  
291 ··<div·role="contentinfo">291 ··<div·role="contentinfo">
292 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.292 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
294 ······</span></p>294 ······</span></p>
295 ··</div>295 ··</div>
  
296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
540 B
html2text {}
    
Offset 216, 9 lines modifiedOffset 216, 9 lines modified
216 ········spider·=·super().from_crawler(crawler,·*args,·**kwargs)216 ········spider·=·super().from_crawler(crawler,·*args,·**kwargs)
217 ········spider._follow_links·=·crawler.settings.getbool(217 ········spider._follow_links·=·crawler.settings.getbool(
218 ············"CRAWLSPIDER_FOLLOW_LINKS",·True218 ············"CRAWLSPIDER_FOLLOW_LINKS",·True
219 ········)219 ········)
220 ········return·spider220 ········return·spider
  
221 ===============================================================================221 ===============================================================================
222 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.222 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
223 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.223 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.18 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spiders/feed.html
    
Offset 282, 15 lines modifiedOffset 282, 15 lines modified
282 ···········</div>282 ···········</div>
283 ··········</div>283 ··········</div>
284 ··········<footer>284 ··········<footer>
  
285 ··<hr/>285 ··<hr/>
  
286 ··<div·role="contentinfo">286 ··<div·role="contentinfo">
287 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.287 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
288 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.288 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
289 ······</span></p>289 ······</span></p>
290 ··</div>290 ··</div>
  
291 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a291 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
292 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>292 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
293 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.293 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
530 B
html2text {}
    
Offset 220, 9 lines modifiedOffset 220, 9 lines modified
220 ················"You·must·define·parse_row·method·in·order·to·scrape·this·CSV220 ················"You·must·define·parse_row·method·in·order·to·scrape·this·CSV
221 feed"221 feed"
222 ············)222 ············)
223 ········response·=·self.adapt_response(response)223 ········response·=·self.adapt_response(response)
224 ········return·self.parse_rows(response)224 ········return·self.parse_rows(response)
  
225 ===============================================================================225 ===============================================================================
226 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.226 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
227 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.227 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/spiders/sitemap.html
    
Offset 238, 15 lines modifiedOffset 238, 15 lines modified
238 ···········</div>238 ···········</div>
239 ··········</div>239 ··········</div>
240 ··········<footer>240 ··········<footer>
  
241 ··<hr/>241 ··<hr/>
  
242 ··<div·role="contentinfo">242 ··<div·role="contentinfo">
243 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.243 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
244 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.244 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
245 ······</span></p>245 ······</span></p>
246 ··</div>246 ··</div>
  
247 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a247 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
248 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>248 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
249 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.249 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
509 B
html2text {}
    
Offset 166, 9 lines modifiedOffset 166, 9 lines modified
166 ········yield·d["loc"]166 ········yield·d["loc"]
  
167 ········#·Also·consider·alternate·URLs·(xhtml:link·rel="alternate")167 ········#·Also·consider·alternate·URLs·(xhtml:link·rel="alternate")
168 ········if·alt·and·"alternate"·in·d:168 ········if·alt·and·"alternate"·in·d:
169 ············yield·from·d["alternate"]169 ············yield·from·d["alternate"]
  
170 ===============================================================================170 ===============================================================================
171 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.171 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
172 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.172 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.13 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/statscollectors.html
    
Offset 218, 15 lines modifiedOffset 218, 15 lines modified
218 ···········</div>218 ···········</div>
219 ··········</div>219 ··········</div>
220 ··········<footer>220 ··········<footer>
  
221 ··<hr/>221 ··<hr/>
  
222 ··<div·role="contentinfo">222 ··<div·role="contentinfo">
223 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.223 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
224 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.224 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
225 ······</span></p>225 ······</span></p>
226 ··</div>226 ··</div>
  
227 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a227 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
228 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>228 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
229 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.229 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
469 B
html2text {}
    
Offset 155, 9 lines modifiedOffset 155, 9 lines modified
155 ····def·max_value(self,·key,·value,·spider=None):155 ····def·max_value(self,·key,·value,·spider=None):
156 ········pass156 ········pass
  
157 ····def·min_value(self,·key,·value,·spider=None):157 ····def·min_value(self,·key,·value,·spider=None):
158 ········pass158 ········pass
  
159 ===============================================================================159 ===============================================================================
160 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.160 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
161 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.161 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.17 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/curl.html
    
Offset 245, 15 lines modifiedOffset 245, 15 lines modified
245 ···········</div>245 ···········</div>
246 ··········</div>246 ··········</div>
247 ··········<footer>247 ··········<footer>
  
248 ··<hr/>248 ··<hr/>
  
249 ··<div·role="contentinfo">249 ··<div·role="contentinfo">
250 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.250 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
251 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.251 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
252 ······</span></p>252 ······</span></p>
253 ··</div>253 ··</div>
  
254 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a254 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
255 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>255 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
256 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.256 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
518 B
html2text {}
    
Offset 170, 9 lines modifiedOffset 170, 9 lines modified
170 ············#·if·the·"data"·is·specified·but·the·"method"·is·not·specified,170 ············#·if·the·"data"·is·specified·but·the·"method"·is·not·specified,
171 ············#·the·default·method·is·'POST'171 ············#·the·default·method·is·'POST'
172 ············result["method"]·=·"POST"172 ············result["method"]·=·"POST"
  
173 ····return·result173 ····return·result
  
174 ===============================================================================174 ===============================================================================
175 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.175 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
176 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.176 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.16 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/defer.html
    
Offset 510, 15 lines modifiedOffset 510, 15 lines modified
510 ···········</div>510 ···········</div>
511 ··········</div>511 ··········</div>
512 ··········<footer>512 ··········<footer>
  
513 ··<hr/>513 ··<hr/>
  
514 ··<div·role="contentinfo">514 ··<div·role="contentinfo">
515 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.515 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
516 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.516 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
517 ······</span></p>517 ······</span></p>
518 ··</div>518 ··</div>
  
519 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a519 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
520 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>520 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
521 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.521 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
510 B
html2text {}
    
Offset 467, 9 lines modifiedOffset 467, 9 lines modified
467 ················extra_response·=·await·maybe_deferred_to_future(d)467 ················extra_response·=·await·maybe_deferred_to_future(d)
468 ····"""468 ····"""
469 ····if·not·is_asyncio_reactor_installed():469 ····if·not·is_asyncio_reactor_installed():
470 ········return·d470 ········return·d
471 ····return·deferred_to_future(d)471 ····return·deferred_to_future(d)
  
472 ===============================================================================472 ===============================================================================
473 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.473 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
474 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.474 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/log.html
    
Offset 368, 15 lines modifiedOffset 368, 15 lines modified
368 ···········</div>368 ···········</div>
369 ··········</div>369 ··········</div>
370 ··········<footer>370 ··········<footer>
  
371 ··<hr/>371 ··<hr/>
  
372 ··<div·role="contentinfo">372 ··<div·role="contentinfo">
373 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.373 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
374 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.374 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
375 ······</span></p>375 ······</span></p>
376 ··</div>376 ··</div>
  
377 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a377 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
378 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>378 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
379 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.379 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
553 B
html2text {}
    
Offset 298, 9 lines modifiedOffset 298, 9 lines modified
298 ····#·NOTE:·This·also·handles·'args'·being·an·empty·dict,·that·case·doesn't298 ····#·NOTE:·This·also·handles·'args'·being·an·empty·dict,·that·case·doesn't
299 ····#·play·well·in·logger.log·calls299 ····#·play·well·in·logger.log·calls
300 ····args·=·logkws·if·not·logkws.get("args")·else·logkws["args"]300 ····args·=·logkws·if·not·logkws.get("args")·else·logkws["args"]
  
301 ····return·(level,·message,·args)301 ····return·(level,·message,·args)
  
302 ===============================================================================302 ===============================================================================
303 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.303 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
304 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.304 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/reactor.html
    
Offset 300, 15 lines modifiedOffset 300, 15 lines modified
300 ···········</div>300 ···········</div>
301 ··········</div>301 ··········</div>
302 ··········<footer>302 ··········<footer>
  
303 ··<hr/>303 ··<hr/>
  
304 ··<div·role="contentinfo">304 ··<div·role="contentinfo">
305 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.305 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
306 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.306 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
307 ······</span></p>307 ······</span></p>
308 ··</div>308 ··</div>
  
309 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a309 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
310 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>310 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
311 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.311 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
491 B
html2text {}
    
Offset 227, 9 lines modifiedOffset 227, 9 lines modified
  
227 def·is_asyncio_reactor_installed():227 def·is_asyncio_reactor_installed():
228 ····from·twisted.internet·import·reactor228 ····from·twisted.internet·import·reactor
  
229 ····return·isinstance(reactor,·asyncioreactor.AsyncioSelectorReactor)229 ····return·isinstance(reactor,·asyncioreactor.AsyncioSelectorReactor)
  
230 ===============================================================================230 ===============================================================================
231 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.231 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
232 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.232 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/request.html
    
Offset 466, 15 lines modifiedOffset 466, 15 lines modified
466 ···········</div>466 ···········</div>
467 ··········</div>467 ··········</div>
468 ··········<footer>468 ··········<footer>
  
469 ··<hr/>469 ··<hr/>
  
470 ··<div·role="contentinfo">470 ··<div·role="contentinfo">
471 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.471 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
472 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.472 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
473 ······</span></p>473 ······</span></p>
474 ··</div>474 ··</div>
  
475 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a475 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
476 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>476 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
477 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.477 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
498 B
html2text {}
    
Offset 402, 9 lines modifiedOffset 402, 9 lines modified
402 ····name·=·str(name)402 ····name·=·str(name)
403 ····try:403 ····try:
404 ········return·getattr(obj,·name)404 ········return·getattr(obj,·name)
405 ····except·AttributeError:405 ····except·AttributeError:
406 ········raise·ValueError(f"Method·{name!r}·not·found·in:·{obj}")406 ········raise·ValueError(f"Method·{name!r}·not·found·in:·{obj}")
  
407 ===============================================================================407 ===============================================================================
408 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.408 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
409 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.409 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.21 KB
./usr/share/doc/python-scrapy-doc/html/_modules/scrapy/utils/trackref.html
    
Offset 202, 15 lines modifiedOffset 202, 15 lines modified
202 ···········</div>202 ···········</div>
203 ··········</div>203 ··········</div>
204 ··········<footer>204 ··········<footer>
  
205 ··<hr/>205 ··<hr/>
  
206 ··<div·role="contentinfo">206 ··<div·role="contentinfo">
207 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.207 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
208 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.208 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
209 ······</span></p>209 ······</span></p>
210 ··</div>210 ··</div>
  
211 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a211 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
212 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>212 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
213 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.213 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
557 B
html2text {}
    
Offset 132, 9 lines modifiedOffset 132, 9 lines modified
132 [docs]def·iter_all(class_name):132 [docs]def·iter_all(class_name):
133 ····"""Iterate·over·all·objects·of·the·same·class·by·its·class·name"""133 ····"""Iterate·over·all·objects·of·the·same·class·by·its·class·name"""
134 ····for·cls,·wdict·in·live_refs.items():134 ····for·cls,·wdict·in·live_refs.items():
135 ········if·cls.__name__·==·class_name:135 ········if·cls.__name__·==·class_name:
136 ············return·wdict.keys()136 ············return·wdict.keys()
  
137 ===============================================================================137 ===============================================================================
138 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.138 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
139 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.139 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.43 KB
./usr/share/doc/python-scrapy-doc/html/contributing.html
    
Offset 404, 15 lines modifiedOffset 404, 15 lines modified
404 ········<a·href="news.html"·class="btn·btn-neutral·float-left"·title="Release·notes"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>404 ········<a·href="news.html"·class="btn·btn-neutral·float-left"·title="Release·notes"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
405 ········<a·href="versioning.html"·class="btn·btn-neutral·float-right"·title="Versioning·and·API·stability"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>405 ········<a·href="versioning.html"·class="btn·btn-neutral·float-right"·title="Versioning·and·API·stability"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
406 ····</div>406 ····</div>
  
407 ··<hr/>407 ··<hr/>
  
408 ··<div·role="contentinfo">408 ··<div·role="contentinfo">
409 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.409 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
410 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.410 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
411 ······</span></p>411 ······</span></p>
412 ··</div>412 ··</div>
  
413 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a413 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
414 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>414 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
415 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.415 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
458 B
html2text {}
    
Offset 265, 9 lines modifiedOffset 265, 9 lines modified
265 example,·the·item·loaders·code·is·in:265 example,·the·item·loaders·code·is·in:
266 scrapy.loader266 scrapy.loader
267 And·their·unit-tests·are·in:267 And·their·unit-tests·are·in:
268 tests/test_loader.py268 tests/test_loader.py
  
269 Previous·Next269 Previous·Next
270 ===============================================================================270 ===============================================================================
271 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.271 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
272 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.272 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.55 KB
./usr/share/doc/python-scrapy-doc/html/faq.html
    
Offset 506, 15 lines modifiedOffset 506, 15 lines modified
506 ········<a·href="topics/telnetconsole.html"·class="btn·btn-neutral·float-left"·title="Telnet·Console"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>506 ········<a·href="topics/telnetconsole.html"·class="btn·btn-neutral·float-left"·title="Telnet·Console"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
507 ········<a·href="topics/debug.html"·class="btn·btn-neutral·float-right"·title="Debugging·Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>507 ········<a·href="topics/debug.html"·class="btn·btn-neutral·float-right"·title="Debugging·Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
508 ····</div>508 ····</div>
  
509 ··<hr/>509 ··<hr/>
  
510 ··<div·role="contentinfo">510 ··<div·role="contentinfo">
511 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.511 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
512 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.512 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
513 ······</span></p>513 ······</span></p>
514 ··</div>514 ··</div>
  
515 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a515 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
516 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>516 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
517 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.517 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
597 B
html2text {}
    
Offset 365, 9 lines modifiedOffset 365, 9 lines modified
365 *****365 *****
366 This·may·happen·if·your·Scrapy·project·has·a·spider·module·with·a·name·that366 This·may·happen·if·your·Scrapy·project·has·a·spider·module·with·a·name·that
367 conflicts·with·the·name·of·one·of·the·Python_standard_library_modules,·such·as367 conflicts·with·the·name·of·one·of·the·Python_standard_library_modules,·such·as
368 csv.py·or·os.py,·or·any·Python_package·that·you·have·installed.·See·issue_2680.368 csv.py·or·os.py,·or·any·Python_package·that·you·have·installed.·See·issue_2680.
  
369 Previous·Next369 Previous·Next
370 ===============================================================================370 ===============================================================================
371 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.371 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
372 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.372 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.25 KB
./usr/share/doc/python-scrapy-doc/html/genindex.html
    
Offset 3813, 15 lines modifiedOffset 3813, 15 lines modified
3813 ···········</div>3813 ···········</div>
3814 ··········</div>3814 ··········</div>
3815 ··········<footer>3815 ··········<footer>
  
3816 ··<hr/>3816 ··<hr/>
  
3817 ··<div·role="contentinfo">3817 ··<div·role="contentinfo">
3818 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.3818 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
3819 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.3819 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
3820 ······</span></p>3820 ······</span></p>
3821 ··</div>3821 ··</div>
  
3822 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a3822 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
3823 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>3823 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
3824 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.3824 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
638 B
html2text {}
    
Offset 1053, 9 lines modifiedOffset 1053, 9 lines modified
1053 ······scrapy.spiders)·················method)1053 ······scrapy.spiders)·················method)
1054 ····*·XmlItemExporter_(class_in···········o·(scrapy.selector.Selector1054 ····*·XmlItemExporter_(class_in···········o·(scrapy.selector.Selector
1055 ······scrapy.exporters)·····················method)1055 ······scrapy.exporters)·····················method)
1056 ····*·XmlResponse_(class_in···············o·(scrapy.selector.SelectorList1056 ····*·XmlResponse_(class_in···············o·(scrapy.selector.SelectorList
1057 ······scrapy.http)··························method)1057 ······scrapy.http)··························method)
  
1058 ===============================================================================1058 ===============================================================================
1059 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.1059 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
1060 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.1060 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.36 KB
./usr/share/doc/python-scrapy-doc/html/index.html
    
Offset 298, 15 lines modifiedOffset 298, 15 lines modified
298 ··········<footer><div·class="rst-footer-buttons"·role="navigation"·aria-label="Footer">298 ··········<footer><div·class="rst-footer-buttons"·role="navigation"·aria-label="Footer">
299 ········<a·href="intro/overview.html"·class="btn·btn-neutral·float-right"·title="Scrapy·at·a·glance"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>299 ········<a·href="intro/overview.html"·class="btn·btn-neutral·float-right"·title="Scrapy·at·a·glance"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
300 ····</div>300 ····</div>
  
301 ··<hr/>301 ··<hr/>
  
302 ··<div·role="contentinfo">302 ··<div·role="contentinfo">
303 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.303 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
304 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.304 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
305 ······</span></p>305 ······</span></p>
306 ··</div>306 ··</div>
  
307 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a307 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
308 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>308 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
309 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.309 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
510 B
html2text {}
    
Offset 184, 9 lines modifiedOffset 184, 9 lines modified
184 ··Contributing_to_Scrapy184 ··Contributing_to_Scrapy
185 ······Learn·how·to·contribute·to·the·Scrapy·project.185 ······Learn·how·to·contribute·to·the·Scrapy·project.
186 ··Versioning_and_API_stability186 ··Versioning_and_API_stability
187 ······Understand·Scrapy·versioning·and·API·stability.187 ······Understand·Scrapy·versioning·and·API·stability.
  
188 Next188 Next
189 ===============================================================================189 ===============================================================================
190 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.190 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
191 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.191 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.61 KB
./usr/share/doc/python-scrapy-doc/html/intro/examples.html
    
Offset 155, 15 lines modifiedOffset 155, 15 lines modified
155 ········<a·href="tutorial.html"·class="btn·btn-neutral·float-left"·title="Scrapy·Tutorial"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>155 ········<a·href="tutorial.html"·class="btn·btn-neutral·float-left"·title="Scrapy·Tutorial"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
156 ········<a·href="../topics/commands.html"·class="btn·btn-neutral·float-right"·title="Command·line·tool"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>156 ········<a·href="../topics/commands.html"·class="btn·btn-neutral·float-right"·title="Command·line·tool"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
157 ····</div>157 ····</div>
  
158 ··<hr/>158 ··<hr/>
  
159 ··<div·role="contentinfo">159 ··<div·role="contentinfo">
160 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.160 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
161 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.161 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
162 ······</span></p>162 ······</span></p>
163 ··</div>163 ··</div>
  
164 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a164 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
165 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>165 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
166 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.166 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
637 B
html2text {}
    
Offset 73, 9 lines modifiedOffset 73, 9 lines modified
73 expressions.73 expressions.
74 The·quotesbot·project·is·available·at:·https://github.com/scrapy/quotesbot.·You74 The·quotesbot·project·is·available·at:·https://github.com/scrapy/quotesbot.·You
75 can·find·more·information·about·it·in·the·project’s·README.75 can·find·more·information·about·it·in·the·project’s·README.
76 If·you’re·familiar·with·git,·you·can·checkout·the·code.·Otherwise·you·can76 If·you’re·familiar·with·git,·you·can·checkout·the·code.·Otherwise·you·can
77 download·the·project·as·a·zip·file·by·clicking·here.77 download·the·project·as·a·zip·file·by·clicking·here.
78 Previous·Next78 Previous·Next
79 ===============================================================================79 ===============================================================================
80 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.80 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
81 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.81 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.49 KB
./usr/share/doc/python-scrapy-doc/html/intro/install.html
    
Offset 389, 15 lines modifiedOffset 389, 15 lines modified
389 ········<a·href="overview.html"·class="btn·btn-neutral·float-left"·title="Scrapy·at·a·glance"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>389 ········<a·href="overview.html"·class="btn·btn-neutral·float-left"·title="Scrapy·at·a·glance"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
390 ········<a·href="tutorial.html"·class="btn·btn-neutral·float-right"·title="Scrapy·Tutorial"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>390 ········<a·href="tutorial.html"·class="btn·btn-neutral·float-right"·title="Scrapy·Tutorial"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
391 ····</div>391 ····</div>
  
392 ··<hr/>392 ··<hr/>
  
393 ··<div·role="contentinfo">393 ··<div·role="contentinfo">
394 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.394 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
395 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.395 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
396 ······</span></p>396 ······</span></p>
397 ··</div>397 ··</div>
  
398 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a398 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
399 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>399 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
400 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.400 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
529 B
html2text {}
    
Offset 240, 9 lines modifiedOffset 240, 9 lines modified
240 To·install·a·version·of·pyOpenSSL·that·your·version·of·Twisted·supports,240 To·install·a·version·of·pyOpenSSL·that·your·version·of·Twisted·supports,
241 reinstall·Twisted·with·the·tls·extra·option:241 reinstall·Twisted·with·the·tls·extra·option:
242 pip·install·twisted[tls]242 pip·install·twisted[tls]
243 For·details,·see·Issue_#2473.243 For·details,·see·Issue_#2473.
  
244 Previous·Next244 Previous·Next
245 ===============================================================================245 ===============================================================================
246 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.246 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
247 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.247 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/intro/overview.html
    
Offset 277, 15 lines modifiedOffset 277, 15 lines modified
277 ········<a·href="../index.html"·class="btn·btn-neutral·float-left"·title="Scrapy·2.8·documentation"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>277 ········<a·href="../index.html"·class="btn·btn-neutral·float-left"·title="Scrapy·2.8·documentation"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
278 ········<a·href="install.html"·class="btn·btn-neutral·float-right"·title="Installation·guide"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>278 ········<a·href="install.html"·class="btn·btn-neutral·float-right"·title="Installation·guide"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
279 ····</div>279 ····</div>
  
280 ··<hr/>280 ··<hr/>
  
281 ··<div·role="contentinfo">281 ··<div·role="contentinfo">
282 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.282 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
283 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.283 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
284 ······</span></p>284 ······</span></p>
285 ··</div>285 ··</div>
  
286 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a286 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
287 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>287 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
288 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.288 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
560 B
html2text {}
    
Offset 175, 9 lines modifiedOffset 175, 9 lines modified
175 *****·What’s·next?¶·*****175 *****·What’s·next?¶·*****
176 The·next·steps·for·you·are·to·install_Scrapy,·follow_through_the_tutorial·to176 The·next·steps·for·you·are·to·install_Scrapy,·follow_through_the_tutorial·to
177 learn·how·to·create·a·full-blown·Scrapy·project·and·join_the_community.·Thanks177 learn·how·to·create·a·full-blown·Scrapy·project·and·join_the_community.·Thanks
178 for·your·interest!178 for·your·interest!
  
179 Previous·Next179 Previous·Next
180 ===============================================================================180 ===============================================================================
181 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.181 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
182 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.182 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.57 KB
./usr/share/doc/python-scrapy-doc/html/intro/tutorial.html
    
Offset 841, 15 lines modifiedOffset 841, 15 lines modified
841 ········<a·href="install.html"·class="btn·btn-neutral·float-left"·title="Installation·guide"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>841 ········<a·href="install.html"·class="btn·btn-neutral·float-left"·title="Installation·guide"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
842 ········<a·href="examples.html"·class="btn·btn-neutral·float-right"·title="Examples"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>842 ········<a·href="examples.html"·class="btn·btn-neutral·float-right"·title="Examples"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
843 ····</div>843 ····</div>
  
844 ··<hr/>844 ··<hr/>
  
845 ··<div·role="contentinfo">845 ··<div·role="contentinfo">
846 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.846 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
847 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.847 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
848 ······</span></p>848 ······</span></p>
849 ··</div>849 ··</div>
  
850 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a850 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
851 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>851 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
852 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.852 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
620 B
html2text {}
    
Offset 644, 9 lines modifiedOffset 644, 9 lines modified
644 You·can·continue·from·the·section·Basic_concepts·to·know·more·about·the644 You·can·continue·from·the·section·Basic_concepts·to·know·more·about·the
645 command-line·tool,·spiders,·selectors·and·other·things·the·tutorial·hasn’t645 command-line·tool,·spiders,·selectors·and·other·things·the·tutorial·hasn’t
646 covered·like·modeling·the·scraped·data.·If·you·prefer·to·play·with·an·example646 covered·like·modeling·the·scraped·data.·If·you·prefer·to·play·with·an·example
647 project,·check·the·Examples·section.647 project,·check·the·Examples·section.
  
648 Previous·Next648 Previous·Next
649 ===============================================================================649 ===============================================================================
650 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.650 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
651 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.651 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.42 KB
./usr/share/doc/python-scrapy-doc/html/news.html
    
Offset 5505, 15 lines modifiedOffset 5505, 15 lines modified
5505 ········<a·href="topics/api.html"·class="btn·btn-neutral·float-left"·title="Core·API"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>5505 ········<a·href="topics/api.html"·class="btn·btn-neutral·float-left"·title="Core·API"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
5506 ········<a·href="contributing.html"·class="btn·btn-neutral·float-right"·title="Contributing·to·Scrapy"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>5506 ········<a·href="contributing.html"·class="btn·btn-neutral·float-right"·title="Contributing·to·Scrapy"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
5507 ····</div>5507 ····</div>
  
5508 ··<hr/>5508 ··<hr/>
  
5509 ··<div·role="contentinfo">5509 ··<div·role="contentinfo">
5510 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.5510 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
5511 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.5511 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
5512 ······</span></p>5512 ······</span></p>
5513 ··</div>5513 ··</div>
  
5514 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a5514 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
5515 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>5515 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
5516 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.5516 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
472 B
html2text {}
    
Offset 4218, 9 lines modifiedOffset 4218, 9 lines modified
4218 ······scrapy.utils.markup.replace_escape_chars·instead·(r1865)4218 ······scrapy.utils.markup.replace_escape_chars·instead·(r1865)
  
4219 *****·Scrapy·0.7¶·*****4219 *****·Scrapy·0.7¶·*****
4220 First·release·of·Scrapy.4220 First·release·of·Scrapy.
  
4221 Previous·Next4221 Previous·Next
4222 ===============================================================================4222 ===============================================================================
4223 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.4223 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
4224 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.4224 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.29 KB
./usr/share/doc/python-scrapy-doc/html/py-modindex.html
    
Offset 421, 15 lines modifiedOffset 421, 15 lines modified
421 ···········</div>421 ···········</div>
422 ··········</div>422 ··········</div>
423 ··········<footer>423 ··········<footer>
  
424 ··<hr/>424 ··<hr/>
  
425 ··<div·role="contentinfo">425 ··<div·role="contentinfo">
426 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.426 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
427 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.427 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
428 ······</span></p>428 ······</span></p>
429 ··</div>429 ··</div>
  
430 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a430 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
431 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>431 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
432 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.432 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
668 B
html2text {}
    
Offset 127, 9 lines modifiedOffset 127, 9 lines modified
127 ····   ·scrapy.spidermiddlewares.urllength····URL·Length·Spider·Middleware127 ····   ·scrapy.spidermiddlewares.urllength····URL·Length·Spider·Middleware
128 ····   ·scrapy.statscollectors················Stats·Collectors128 ····   ·scrapy.statscollectors················Stats·Collectors
129 ····   ·scrapy.utils.log······················Logging·utils129 ····   ·scrapy.utils.log······················Logging·utils
130 ····   ·scrapy.utils.trackref·················Track·references·of·live130 ····   ·scrapy.utils.trackref·················Track·references·of·live
131 ·················································objects131 ·················································objects
  
132 ===============================================================================132 ===============================================================================
133 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.133 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
134 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.134 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.15 KB
./usr/share/doc/python-scrapy-doc/html/search.html
    
Offset 149, 15 lines modifiedOffset 149, 15 lines modified
149 ···········</div>149 ···········</div>
150 ··········</div>150 ··········</div>
151 ··········<footer>151 ··········<footer>
  
152 ··<hr/>152 ··<hr/>
  
153 ··<div·role="contentinfo">153 ··<div·role="contentinfo">
154 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.154 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
155 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.155 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
156 ······</span></p>156 ······</span></p>
157 ··</div>157 ··</div>
  
158 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a158 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
159 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>159 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
160 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.160 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
541 B
html2text {}
    
Offset 61, 9 lines modifiedOffset 61, 9 lines modified
61 ····*·Versioning_and_API_stability61 ····*·Versioning_and_API_stability
62 ···Scrapy62 ···Scrapy
63 ····*·Search63 ····*·Search
64 ===============================================================================64 ===============================================================================
65 Please·activate·JavaScript·to·enable·the·search·functionality.65 Please·activate·JavaScript·to·enable·the·search·functionality.
  
66 ===============================================================================66 ===============================================================================
67 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.67 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
68 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.68 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.52 KB
./usr/share/doc/python-scrapy-doc/html/topics/api.html
    
Offset 1036, 15 lines modifiedOffset 1036, 15 lines modified
1036 ········<a·href="components.html"·class="btn·btn-neutral·float-left"·title="Components"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1036 ········<a·href="components.html"·class="btn·btn-neutral·float-left"·title="Components"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1037 ········<a·href="../news.html"·class="btn·btn-neutral·float-right"·title="Release·notes"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1037 ········<a·href="../news.html"·class="btn·btn-neutral·float-right"·title="Release·notes"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1038 ····</div>1038 ····</div>
  
1039 ··<hr/>1039 ··<hr/>
  
1040 ··<div·role="contentinfo">1040 ··<div·role="contentinfo">
1041 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1041 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1042 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1042 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1043 ······</span></p>1043 ······</span></p>
1044 ··</div>1044 ··</div>
  
1045 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1045 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1046 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1046 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1047 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1047 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
574 B
html2text {}
    
Offset 564, 9 lines modifiedOffset 564, 9 lines modified
564 ············Open·the·given·spider·for·stats·collection.564 ············Open·the·given·spider·for·stats·collection.
565 ········close_spider(spider)[source]¶565 ········close_spider(spider)[source]¶
566 ············Close·the·given·spider.·After·this·is·called,·no·more·specific566 ············Close·the·given·spider.·After·this·is·called,·no·more·specific
567 ············stats·can·be·accessed·or·collected.567 ············stats·can·be·accessed·or·collected.
  
568 Previous·Next568 Previous·Next
569 ===============================================================================569 ===============================================================================
570 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.570 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
571 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.571 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.47 KB
./usr/share/doc/python-scrapy-doc/html/topics/architecture.html
    
Offset 287, 15 lines modifiedOffset 287, 15 lines modified
287 ········<a·href="asyncio.html"·class="btn·btn-neutral·float-left"·title="asyncio"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>287 ········<a·href="asyncio.html"·class="btn·btn-neutral·float-left"·title="asyncio"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
288 ········<a·href="downloader-middleware.html"·class="btn·btn-neutral·float-right"·title="Downloader·Middleware"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>288 ········<a·href="downloader-middleware.html"·class="btn·btn-neutral·float-right"·title="Downloader·Middleware"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
289 ····</div>289 ····</div>
  
290 ··<hr/>290 ··<hr/>
  
291 ··<div·role="contentinfo">291 ··<div·role="contentinfo">
292 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.292 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.293 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
294 ······</span></p>294 ······</span></p>
295 ··</div>295 ··</div>
  
296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a296 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>297 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.298 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
484 B
html2text {}
    
Offset 169, 9 lines modifiedOffset 169, 9 lines modified
169 links:169 links:
170 ····*·twisted:core/howto/defer-intro170 ····*·twisted:core/howto/defer-intro
171 ····*·Twisted_-_hello,_asynchronous_programming171 ····*·Twisted_-_hello,_asynchronous_programming
172 ····*·Twisted_Introduction_-_Krondo172 ····*·Twisted_Introduction_-_Krondo
  
173 Previous·Next173 Previous·Next
174 ===============================================================================174 ===============================================================================
175 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.175 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
176 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.176 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/asyncio.html
    
Offset 300, 15 lines modifiedOffset 300, 15 lines modified
300 ········<a·href="coroutines.html"·class="btn·btn-neutral·float-left"·title="Coroutines"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>300 ········<a·href="coroutines.html"·class="btn·btn-neutral·float-left"·title="Coroutines"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
301 ········<a·href="architecture.html"·class="btn·btn-neutral·float-right"·title="Architecture·overview"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>301 ········<a·href="architecture.html"·class="btn·btn-neutral·float-right"·title="Architecture·overview"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
302 ····</div>302 ····</div>
  
303 ··<hr/>303 ··<hr/>
  
304 ··<div·role="contentinfo">304 ··<div·role="contentinfo">
305 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.305 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
306 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.306 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
307 ······</span></p>307 ······</span></p>
308 ··</div>308 ··</div>
  
309 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a309 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
310 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>310 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
311 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.311 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
565 B
html2text {}
    
Offset 167, 9 lines modifiedOffset 167, 9 lines modified
167 ················f"reactor.·Make·sure·you·have·it·configured·in·the·"167 ················f"reactor.·Make·sure·you·have·it·configured·in·the·"
168 ················f"TWISTED_REACTOR·setting.·See·the·asyncio·documentation·"168 ················f"TWISTED_REACTOR·setting.·See·the·asyncio·documentation·"
169 ················f"of·Scrapy·for·more·information."169 ················f"of·Scrapy·for·more·information."
170 ············)170 ············)
  
171 Previous·Next171 Previous·Next
172 ===============================================================================172 ===============================================================================
173 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.173 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
174 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.174 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.5 KB
./usr/share/doc/python-scrapy-doc/html/topics/autothrottle.html
    
Offset 285, 15 lines modifiedOffset 285, 15 lines modified
285 ········<a·href="deploy.html"·class="btn·btn-neutral·float-left"·title="Deploying·Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>285 ········<a·href="deploy.html"·class="btn·btn-neutral·float-left"·title="Deploying·Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
286 ········<a·href="benchmarking.html"·class="btn·btn-neutral·float-right"·title="Benchmarking"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>286 ········<a·href="benchmarking.html"·class="btn·btn-neutral·float-right"·title="Benchmarking"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
287 ····</div>287 ····</div>
  
288 ··<hr/>288 ··<hr/>
  
289 ··<div·role="contentinfo">289 ··<div·role="contentinfo">
290 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.290 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
291 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.291 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
292 ······</span></p>292 ······</span></p>
293 ··</div>293 ··</div>
  
294 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a294 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
295 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>295 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
296 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.296 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
533 B
html2text {}
    
Offset 174, 9 lines modifiedOffset 174, 9 lines modified
174 Default:·False174 Default:·False
175 Enable·AutoThrottle·debug·mode·which·will·display·stats·on·every·response175 Enable·AutoThrottle·debug·mode·which·will·display·stats·on·every·response
176 received,·so·you·can·see·how·the·throttling·parameters·are·being·adjusted·in176 received,·so·you·can·see·how·the·throttling·parameters·are·being·adjusted·in
177 real·time.177 real·time.
  
178 Previous·Next178 Previous·Next
179 ===============================================================================179 ===============================================================================
180 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.180 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
181 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.181 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.66 KB
./usr/share/doc/python-scrapy-doc/html/topics/benchmarking.html
    
Offset 221, 15 lines modifiedOffset 221, 15 lines modified
221 ········<a·href="autothrottle.html"·class="btn·btn-neutral·float-left"·title="AutoThrottle·extension"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>221 ········<a·href="autothrottle.html"·class="btn·btn-neutral·float-left"·title="AutoThrottle·extension"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
222 ········<a·href="jobs.html"·class="btn·btn-neutral·float-right"·title="Jobs:·pausing·and·resuming·crawls"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>222 ········<a·href="jobs.html"·class="btn·btn-neutral·float-right"·title="Jobs:·pausing·and·resuming·crawls"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
223 ····</div>223 ····</div>
  
224 ··<hr/>224 ··<hr/>
  
225 ··<div·role="contentinfo">225 ··<div·role="contentinfo">
226 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.226 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
227 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.227 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
228 ······</span></p>228 ······</span></p>
229 ··</div>229 ··</div>
  
230 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a230 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
231 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>231 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
232 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.232 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
665 B
html2text {}
    
Offset 153, 9 lines modifiedOffset 153, 9 lines modified
153 hardware·where·you·run·it.·Note·that·this·is·a·very·simple·spider·intended·to153 hardware·where·you·run·it.·Note·that·this·is·a·very·simple·spider·intended·to
154 follow·links,·any·custom·spider·you·write·will·probably·do·more·stuff·which154 follow·links,·any·custom·spider·you·write·will·probably·do·more·stuff·which
155 results·in·slower·crawl·rates.·How·slower·depends·on·how·much·your·spider·does155 results·in·slower·crawl·rates.·How·slower·depends·on·how·much·your·spider·does
156 and·how·well·it’s·written.156 and·how·well·it’s·written.
157 Use·scrapy-bench·for·more·complex·benchmarking.157 Use·scrapy-bench·for·more·complex·benchmarking.
158 Previous·Next158 Previous·Next
159 ===============================================================================159 ===============================================================================
160 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.160 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
161 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.161 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.57 KB
./usr/share/doc/python-scrapy-doc/html/topics/broad-crawls.html
    
Offset 339, 15 lines modifiedOffset 339, 15 lines modified
339 ········<a·href="practices.html"·class="btn·btn-neutral·float-left"·title="Common·Practices"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>339 ········<a·href="practices.html"·class="btn·btn-neutral·float-left"·title="Common·Practices"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
340 ········<a·href="developer-tools.html"·class="btn·btn-neutral·float-right"·title="Using·your·browser’s·Developer·Tools·for·scraping"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>340 ········<a·href="developer-tools.html"·class="btn·btn-neutral·float-right"·title="Using·your·browser’s·Developer·Tools·for·scraping"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
341 ····</div>341 ····</div>
  
342 ··<hr/>342 ··<hr/>
  
343 ··<div·role="contentinfo">343 ··<div·role="contentinfo">
344 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.344 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
345 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.345 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
346 ······</span></p>346 ······</span></p>
347 ··</div>347 ··</div>
  
348 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a348 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
349 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>349 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
350 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.350 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
555 B
html2text {}
    
Offset 219, 9 lines modifiedOffset 219, 9 lines modified
  
219 *****·Install·a·specific·Twisted·reactor¶·*****219 *****·Install·a·specific·Twisted·reactor¶·*****
220 If·the·crawl·is·exceeding·the·system’s·capabilities,·you·might·want·to·try220 If·the·crawl·is·exceeding·the·system’s·capabilities,·you·might·want·to·try
221 installing·a·specific·Twisted·reactor,·via·the·TWISTED_REACTOR·setting.221 installing·a·specific·Twisted·reactor,·via·the·TWISTED_REACTOR·setting.
  
222 Previous·Next222 Previous·Next
223 ===============================================================================223 ===============================================================================
224 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.224 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
225 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.225 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.39 KB
./usr/share/doc/python-scrapy-doc/html/topics/commands.html
    
Offset 714, 15 lines modifiedOffset 714, 15 lines modified
714 ········<a·href="../intro/examples.html"·class="btn·btn-neutral·float-left"·title="Examples"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>714 ········<a·href="../intro/examples.html"·class="btn·btn-neutral·float-left"·title="Examples"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
715 ········<a·href="spiders.html"·class="btn·btn-neutral·float-right"·title="Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>715 ········<a·href="spiders.html"·class="btn·btn-neutral·float-right"·title="Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
716 ····</div>716 ····</div>
  
717 ··<hr/>717 ··<hr/>
  
718 ··<div·role="contentinfo">718 ··<div·role="contentinfo">
719 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.719 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
720 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.720 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
721 ······</span></p>721 ······</span></p>
722 ··</div>722 ··</div>
  
723 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a723 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
724 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>724 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
725 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.725 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
427 B
html2text {}
    
Offset 516, 9 lines modifiedOffset 516, 9 lines modified
516 ······'my_command=my_scrapy_module.commands:MyCommand',516 ······'my_command=my_scrapy_module.commands:MyCommand',
517 ····],517 ····],
518 ··},518 ··},
519 ·)519 ·)
  
520 Previous·Next520 Previous·Next
521 ===============================================================================521 ===============================================================================
522 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.522 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
523 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.523 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.5 KB
./usr/share/doc/python-scrapy-doc/html/topics/components.html
    
Offset 205, 15 lines modifiedOffset 205, 15 lines modified
205 ········<a·href="exporters.html"·class="btn·btn-neutral·float-left"·title="Item·Exporters"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>205 ········<a·href="exporters.html"·class="btn·btn-neutral·float-left"·title="Item·Exporters"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
206 ········<a·href="api.html"·class="btn·btn-neutral·float-right"·title="Core·API"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>206 ········<a·href="api.html"·class="btn·btn-neutral·float-right"·title="Core·API"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
207 ····</div>207 ····</div>
  
208 ··<hr/>208 ··<hr/>
  
209 ··<div·role="contentinfo">209 ··<div·role="contentinfo">
210 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.210 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.211 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
212 ······</span></p>212 ······</span></p>
213 ··</div>213 ··</div>
  
214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a214 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>215 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.216 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
542 B
html2text {}
    
Offset 115, 9 lines modifiedOffset 115, 9 lines modified
115 ················f"later,·which·allow·defining·the·process_spider_output·"115 ················f"later,·which·allow·defining·the·process_spider_output·"
116 ················f"method·of·spider·middlewares·as·an·asynchronous·"116 ················f"method·of·spider·middlewares·as·an·asynchronous·"
117 ················f"generator."117 ················f"generator."
118 ············)118 ············)
  
119 Previous·Next119 Previous·Next
120 ===============================================================================120 ===============================================================================
121 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.121 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
122 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.122 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.46 KB
./usr/share/doc/python-scrapy-doc/html/topics/contracts.html
    
Offset 319, 15 lines modifiedOffset 319, 15 lines modified
319 ········<a·href="debug.html"·class="btn·btn-neutral·float-left"·title="Debugging·Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>319 ········<a·href="debug.html"·class="btn·btn-neutral·float-left"·title="Debugging·Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
320 ········<a·href="practices.html"·class="btn·btn-neutral·float-right"·title="Common·Practices"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>320 ········<a·href="practices.html"·class="btn·btn-neutral·float-right"·title="Common·Practices"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
321 ····</div>321 ····</div>
  
322 ··<hr/>322 ··<hr/>
  
323 ··<div·role="contentinfo">323 ··<div·role="contentinfo">
324 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.324 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
325 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.325 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
326 ······</span></p>326 ······</span></p>
327 ··</div>327 ··</div>
  
328 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a328 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
329 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>329 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
330 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.330 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
496 B
html2text {}
    
Offset 172, 9 lines modifiedOffset 172, 9 lines modified
  
172 ····def·__init__(self):172 ····def·__init__(self):
173 ········if·os.environ.get('SCRAPY_CHECK'):173 ········if·os.environ.get('SCRAPY_CHECK'):
174 ············pass··#·Do·some·scraper·adjustments·when·a·check·is·running174 ············pass··#·Do·some·scraper·adjustments·when·a·check·is·running
  
175 Previous·Next175 Previous·Next
176 ===============================================================================176 ===============================================================================
177 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.177 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
178 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.178 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.6 KB
./usr/share/doc/python-scrapy-doc/html/topics/coroutines.html
    
Offset 350, 15 lines modifiedOffset 350, 15 lines modified
350 ········<a·href="jobs.html"·class="btn·btn-neutral·float-left"·title="Jobs:·pausing·and·resuming·crawls"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>350 ········<a·href="jobs.html"·class="btn·btn-neutral·float-left"·title="Jobs:·pausing·and·resuming·crawls"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
351 ········<a·href="asyncio.html"·class="btn·btn-neutral·float-right"·title="asyncio"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>351 ········<a·href="asyncio.html"·class="btn·btn-neutral·float-right"·title="asyncio"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
352 ····</div>352 ····</div>
  
353 ··<hr/>353 ··<hr/>
  
354 ··<div·role="contentinfo">354 ··<div·role="contentinfo">
355 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.355 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
356 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.356 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
357 ······</span></p>357 ······</span></p>
358 ··</div>358 ··</div>
  
359 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a359 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
360 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>360 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
361 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.361 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
627 B
html2text {}
    
Offset 212, 9 lines modifiedOffset 212, 9 lines modified
212 In·some·future·version·of·Scrapy,·however,·this·feature·will·be·deprecated·and,212 In·some·future·version·of·Scrapy,·however,·this·feature·will·be·deprecated·and,
213 eventually,·in·a·later·version·of·Scrapy,·this·feature·will·be·removed,·and·all213 eventually,·in·a·later·version·of·Scrapy,·this·feature·will·be·removed,·and·all
214 spider·middlewares·will·be·expected·to·define·their·process_spider_output214 spider·middlewares·will·be·expected·to·define·their·process_spider_output
215 method·as·an·asynchronous·generator.215 method·as·an·asynchronous·generator.
  
216 Previous·Next216 Previous·Next
217 ===============================================================================217 ===============================================================================
218 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.218 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
219 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.219 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.44 KB
./usr/share/doc/python-scrapy-doc/html/topics/debug.html
    
Offset 305, 15 lines modifiedOffset 305, 15 lines modified
305 ········<a·href="../faq.html"·class="btn·btn-neutral·float-left"·title="Frequently·Asked·Questions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>305 ········<a·href="../faq.html"·class="btn·btn-neutral·float-left"·title="Frequently·Asked·Questions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
306 ········<a·href="contracts.html"·class="btn·btn-neutral·float-right"·title="Spiders·Contracts"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>306 ········<a·href="contracts.html"·class="btn·btn-neutral·float-right"·title="Spiders·Contracts"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
307 ····</div>307 ····</div>
  
308 ··<hr/>308 ··<hr/>
  
309 ··<div·role="contentinfo">309 ··<div·role="contentinfo">
310 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.310 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
311 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.311 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
312 ······</span></p>312 ······</span></p>
313 ··</div>313 ··</div>
  
314 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a314 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
315 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>315 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
316 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.316 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
466 B
html2text {}
    
Offset 201, 9 lines modifiedOffset 201, 9 lines modified
201 ····]201 ····]
202 }202 }
203 Also,·make·sure·you·enable·â€œUser·Uncaught·Exceptions”,·to·catch·exceptions203 Also,·make·sure·you·enable·â€œUser·Uncaught·Exceptions”,·to·catch·exceptions
204 in·your·Scrapy·spider.204 in·your·Scrapy·spider.
  
205 Previous·Next205 Previous·Next
206 ===============================================================================206 ===============================================================================
207 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.207 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
208 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.208 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.6 KB
./usr/share/doc/python-scrapy-doc/html/topics/deploy.html
    
Offset 182, 15 lines modifiedOffset 182, 15 lines modified
182 ········<a·href="media-pipeline.html"·class="btn·btn-neutral·float-left"·title="Downloading·and·processing·files·and·images"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>182 ········<a·href="media-pipeline.html"·class="btn·btn-neutral·float-left"·title="Downloading·and·processing·files·and·images"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
183 ········<a·href="autothrottle.html"·class="btn·btn-neutral·float-right"·title="AutoThrottle·extension"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>183 ········<a·href="autothrottle.html"·class="btn·btn-neutral·float-right"·title="AutoThrottle·extension"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
184 ····</div>184 ····</div>
  
185 ··<hr/>185 ··<hr/>
  
186 ··<div·role="contentinfo">186 ··<div·role="contentinfo">
187 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.187 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
188 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.188 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
189 ······</span></p>189 ······</span></p>
190 ··</div>190 ··</div>
  
191 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a191 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
192 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>192 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
193 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.193 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
595 B
html2text {}
    
Offset 94, 9 lines modifiedOffset 94, 9 lines modified
94 Please·refer·to·the·Zyte_Scrapy_Cloud_documentation·for·more·information.94 Please·refer·to·the·Zyte_Scrapy_Cloud_documentation·for·more·information.
95 Zyte·Scrapy·Cloud·is·compatible·with·Scrapyd·and·one·can·switch·between·them·as95 Zyte·Scrapy·Cloud·is·compatible·with·Scrapyd·and·one·can·switch·between·them·as
96 needed·-·the·configuration·is·read·from·the·scrapy.cfg·file·just·like·scrapyd-96 needed·-·the·configuration·is·read·from·the·scrapy.cfg·file·just·like·scrapyd-
97 deploy.97 deploy.
  
98 Previous·Next98 Previous·Next
99 ===============================================================================99 ===============================================================================
100 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.100 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
101 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.101 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.61 KB
./usr/share/doc/python-scrapy-doc/html/topics/developer-tools.html
    
Offset 403, 15 lines modifiedOffset 403, 15 lines modified
403 ········<a·href="broad-crawls.html"·class="btn·btn-neutral·float-left"·title="Broad·Crawls"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>403 ········<a·href="broad-crawls.html"·class="btn·btn-neutral·float-left"·title="Broad·Crawls"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
404 ········<a·href="dynamic-content.html"·class="btn·btn-neutral·float-right"·title="Selecting·dynamically-loaded·content"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>404 ········<a·href="dynamic-content.html"·class="btn·btn-neutral·float-right"·title="Selecting·dynamically-loaded·content"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
405 ····</div>405 ····</div>
  
406 ··<hr/>406 ··<hr/>
  
407 ··<div·role="contentinfo">407 ··<div·role="contentinfo">
408 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.408 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
409 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.409 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
410 ······</span></p>410 ······</span></p>
411 ··</div>411 ··</div>
  
412 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a412 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
413 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>413 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
414 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.414 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
611 B
html2text {}
    
Offset 285, 9 lines modifiedOffset 285, 9 lines modified
285 easily·replicate·the·dynamic·requests·of·the·scrolling·functionality·of·the285 easily·replicate·the·dynamic·requests·of·the·scrolling·functionality·of·the
286 page.·Crawling·dynamic·pages·can·be·quite·daunting·and·pages·can·be·very286 page.·Crawling·dynamic·pages·can·be·quite·daunting·and·pages·can·be·very
287 complex,·but·it·(mostly)·boils·down·to·identifying·the·correct·request·and287 complex,·but·it·(mostly)·boils·down·to·identifying·the·correct·request·and
288 replicating·it·in·your·spider.288 replicating·it·in·your·spider.
  
289 Previous·Next289 Previous·Next
290 ===============================================================================290 ===============================================================================
291 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.291 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
292 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.292 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/topics/djangoitem.html
    
Offset 145, 15 lines modifiedOffset 145, 15 lines modified
145 ···········</div>145 ···········</div>
146 ··········</div>146 ··········</div>
147 ··········<footer>147 ··········<footer>
  
148 ··<hr/>148 ··<hr/>
  
149 ··<div·role="contentinfo">149 ··<div·role="contentinfo">
150 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.150 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
151 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.151 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
152 ······</span></p>152 ······</span></p>
153 ··</div>153 ··</div>
  
154 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a154 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
155 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>155 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
156 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.156 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
572 B
html2text {}
    
Offset 65, 9 lines modifiedOffset 65, 9 lines modified
65 ===============================================================================65 ===============================================================================
66 ******·DjangoItem¶·******66 ******·DjangoItem¶·******
67 DjangoItem·has·been·moved·into·a·separate·project.67 DjangoItem·has·been·moved·into·a·separate·project.
68 It·is·hosted·at:68 It·is·hosted·at:
69 ·····https://github.com/scrapy-plugins/scrapy-djangoitem69 ·····https://github.com/scrapy-plugins/scrapy-djangoitem
  
70 ===============================================================================70 ===============================================================================
71 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.71 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
72 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.72 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.49 KB
./usr/share/doc/python-scrapy-doc/html/topics/downloader-middleware.html
    
Offset 1277, 15 lines modifiedOffset 1277, 15 lines modified
1277 ········<a·href="architecture.html"·class="btn·btn-neutral·float-left"·title="Architecture·overview"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1277 ········<a·href="architecture.html"·class="btn·btn-neutral·float-left"·title="Architecture·overview"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1278 ········<a·href="spider-middleware.html"·class="btn·btn-neutral·float-right"·title="Spider·Middleware"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1278 ········<a·href="spider-middleware.html"·class="btn·btn-neutral·float-right"·title="Spider·Middleware"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1279 ····</div>1279 ····</div>
  
1280 ··<hr/>1280 ··<hr/>
  
1281 ··<div·role="contentinfo">1281 ··<div·role="contentinfo">
1282 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1282 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1283 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1283 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1284 ······</span></p>1284 ······</span></p>
1285 ··</div>1285 ··</div>
  
1286 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1286 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1287 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1287 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1288 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1288 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
478 B
html2text {}
    
Offset 852, 9 lines modifiedOffset 852, 9 lines modified
  
852 **·HTTPPROXY_AUTH_ENCODING¶·**852 **·HTTPPROXY_AUTH_ENCODING¶·**
853 Default:·"latin-1"853 Default:·"latin-1"
854 The·default·encoding·for·proxy·authentication·on·HttpProxyMiddleware.854 The·default·encoding·for·proxy·authentication·on·HttpProxyMiddleware.
  
855 Previous·Next855 Previous·Next
856 ===============================================================================856 ===============================================================================
857 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.857 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
858 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.858 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.61 KB
./usr/share/doc/python-scrapy-doc/html/topics/dynamic-content.html
    
Offset 369, 15 lines modifiedOffset 369, 15 lines modified
369 ········<a·href="developer-tools.html"·class="btn·btn-neutral·float-left"·title="Using·your·browser’s·Developer·Tools·for·scraping"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>369 ········<a·href="developer-tools.html"·class="btn·btn-neutral·float-left"·title="Using·your·browser’s·Developer·Tools·for·scraping"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
370 ········<a·href="leaks.html"·class="btn·btn-neutral·float-right"·title="Debugging·memory·leaks"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>370 ········<a·href="leaks.html"·class="btn·btn-neutral·float-right"·title="Debugging·memory·leaks"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
371 ····</div>371 ····</div>
  
372 ··<hr/>372 ··<hr/>
  
373 ··<div·role="contentinfo">373 ··<div·role="contentinfo">
374 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.374 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
375 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.375 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
376 ······</span></p>376 ······</span></p>
377 ··</div>377 ··</div>
  
378 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a378 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
379 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>379 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
380 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.380 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
595 B
html2text {}
    
Offset 240, 9 lines modifiedOffset 240, 9 lines modified
240 ············return·{"title":·title}240 ············return·{"title":·title}
241 However,·using·playwright-python·directly·as·in·the·above·example·circumvents241 However,·using·playwright-python·directly·as·in·the·above·example·circumvents
242 most·of·the·Scrapy·components·(middlewares,·dupefilter,·etc).·We·recommend242 most·of·the·Scrapy·components·(middlewares,·dupefilter,·etc).·We·recommend
243 using·scrapy-playwright·for·a·better·integration.243 using·scrapy-playwright·for·a·better·integration.
  
244 Previous·Next244 Previous·Next
245 ===============================================================================245 ===============================================================================
246 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.246 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
247 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.247 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.41 KB
./usr/share/doc/python-scrapy-doc/html/topics/email.html
    
Offset 294, 15 lines modifiedOffset 294, 15 lines modified
294 ········<a·href="stats.html"·class="btn·btn-neutral·float-left"·title="Stats·Collection"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>294 ········<a·href="stats.html"·class="btn·btn-neutral·float-left"·title="Stats·Collection"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
295 ········<a·href="telnetconsole.html"·class="btn·btn-neutral·float-right"·title="Telnet·Console"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>295 ········<a·href="telnetconsole.html"·class="btn·btn-neutral·float-right"·title="Telnet·Console"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
296 ····</div>296 ····</div>
  
297 ··<hr/>297 ··<hr/>
  
298 ··<div·role="contentinfo">298 ··<div·role="contentinfo">
299 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.299 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
300 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.300 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
301 ······</span></p>301 ······</span></p>
302 ··</div>302 ··</div>
  
303 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a303 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
304 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>304 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
305 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.305 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
446 B
html2text {}
    
Offset 172, 9 lines modifiedOffset 172, 9 lines modified
  
172 ****·MAIL_SSL¶·****172 ****·MAIL_SSL¶·****
173 Default:·False173 Default:·False
174 Enforce·connecting·using·an·SSL·encrypted·connection174 Enforce·connecting·using·an·SSL·encrypted·connection
  
175 Previous·Next175 Previous·Next
176 ===============================================================================176 ===============================================================================
177 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.177 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
178 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.178 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.51 KB
./usr/share/doc/python-scrapy-doc/html/topics/exceptions.html
    
Offset 297, 15 lines modifiedOffset 297, 15 lines modified
297 ········<a·href="settings.html"·class="btn·btn-neutral·float-left"·title="Settings"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>297 ········<a·href="settings.html"·class="btn·btn-neutral·float-left"·title="Settings"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
298 ········<a·href="logging.html"·class="btn·btn-neutral·float-right"·title="Logging"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>298 ········<a·href="logging.html"·class="btn·btn-neutral·float-right"·title="Logging"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
299 ····</div>299 ····</div>
  
300 ··<hr/>300 ··<hr/>
  
301 ··<div·role="contentinfo">301 ··<div·role="contentinfo">
302 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.302 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
303 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.303 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
304 ······</span></p>304 ······</span></p>
305 ··</div>305 ··</div>
  
306 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a306 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
307 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>307 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
308 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.308 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
564 B
html2text {}
    
Offset 146, 9 lines modifiedOffset 146, 9 lines modified
146 StopDownload(True)·will·raise·a·TypeError.146 StopDownload(True)·will·raise·a·TypeError.
147 See·the·documentation·for·the·bytes_received·and·headers_received·signals·and147 See·the·documentation·for·the·bytes_received·and·headers_received·signals·and
148 the·Stopping_the_download_of_a_Response·topic·for·additional·information·and148 the·Stopping_the_download_of_a_Response·topic·for·additional·information·and
149 examples.149 examples.
  
150 Previous·Next150 Previous·Next
151 ===============================================================================151 ===============================================================================
152 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.152 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
153 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.153 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/exporters.html
    
Offset 652, 15 lines modifiedOffset 652, 15 lines modified
652 ········<a·href="scheduler.html"·class="btn·btn-neutral·float-left"·title="Scheduler"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>652 ········<a·href="scheduler.html"·class="btn·btn-neutral·float-left"·title="Scheduler"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
653 ········<a·href="components.html"·class="btn·btn-neutral·float-right"·title="Components"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>653 ········<a·href="components.html"·class="btn·btn-neutral·float-right"·title="Components"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
654 ····</div>654 ····</div>
  
655 ··<hr/>655 ··<hr/>
  
656 ··<div·role="contentinfo">656 ··<div·role="contentinfo">
657 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.657 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
658 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.658 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
659 ······</span></p>659 ······</span></p>
660 ··</div>660 ··</div>
  
661 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a661 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
662 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>662 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
663 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.663 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
569 B
html2text {}
    
Offset 407, 9 lines modifiedOffset 407, 9 lines modified
407 ········Parameters:407 ········Parameters:
408 ············file·â€“·The·file-like·object·to·use·for·exporting·the·data.·Its408 ············file·â€“·The·file-like·object·to·use·for·exporting·the·data.·Its
409 ············write·method·should·accept·bytes·(a·disk·file·opened·in·binary409 ············write·method·should·accept·bytes·(a·disk·file·opened·in·binary
410 ············mode,·a·BytesIO·object,·etc)410 ············mode,·a·BytesIO·object,·etc)
  
411 Previous·Next411 Previous·Next
412 ===============================================================================412 ===============================================================================
413 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.413 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
414 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.414 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/extensions.html
    
Offset 497, 15 lines modifiedOffset 497, 15 lines modified
497 ········<a·href="spider-middleware.html"·class="btn·btn-neutral·float-left"·title="Spider·Middleware"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>497 ········<a·href="spider-middleware.html"·class="btn·btn-neutral·float-left"·title="Spider·Middleware"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
498 ········<a·href="signals.html"·class="btn·btn-neutral·float-right"·title="Signals"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>498 ········<a·href="signals.html"·class="btn·btn-neutral·float-right"·title="Signals"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
499 ····</div>499 ····</div>
  
500 ··<hr/>500 ··<hr/>
  
501 ··<div·role="contentinfo">501 ··<div·role="contentinfo">
502 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.502 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
503 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.503 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
504 ······</span></p>504 ······</span></p>
505 ··</div>505 ··</div>
  
506 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a506 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
507 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>507 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
508 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.508 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
560 B
html2text {}
    
Offset 311, 9 lines modifiedOffset 311, 9 lines modified
311 is·received.·After·the·debugger·is·exited,·the·Scrapy·process·continues·running311 is·received.·After·the·debugger·is·exited,·the·Scrapy·process·continues·running
312 normally.312 normally.
313 For·more·info·see·Debugging_in_Python.313 For·more·info·see·Debugging_in_Python.
314 This·extension·only·works·on·POSIX-compliant·platforms·(i.e.·not·Windows).314 This·extension·only·works·on·POSIX-compliant·platforms·(i.e.·not·Windows).
  
315 Previous·Next315 Previous·Next
316 ===============================================================================316 ===============================================================================
317 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.317 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
318 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.318 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/feed-exports.html
    
Offset 909, 15 lines modifiedOffset 909, 15 lines modified
909 ········<a·href="item-pipeline.html"·class="btn·btn-neutral·float-left"·title="Item·Pipeline"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>909 ········<a·href="item-pipeline.html"·class="btn·btn-neutral·float-left"·title="Item·Pipeline"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
910 ········<a·href="request-response.html"·class="btn·btn-neutral·float-right"·title="Requests·and·Responses"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>910 ········<a·href="request-response.html"·class="btn·btn-neutral·float-right"·title="Requests·and·Responses"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
911 ····</div>911 ····</div>
  
912 ··<hr/>912 ··<hr/>
  
913 ··<div·role="contentinfo">913 ··<div·role="contentinfo">
914 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.914 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
915 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.915 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
916 ······</span></p>916 ······</span></p>
917 ··</div>917 ··</div>
  
918 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a918 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
919 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>919 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
920 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.920 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
543 B
html2text {}
    
Offset 573, 9 lines modifiedOffset 573, 9 lines modified
573 ······#·myproject/settings.py573 ······#·myproject/settings.py
574 ······FEED_URI_PARAMS·=·'myproject.utils.uri_params'574 ······FEED_URI_PARAMS·=·'myproject.utils.uri_params'
575 ···3.·Use·%(spider_name)s·in·your·feed·URI:575 ···3.·Use·%(spider_name)s·in·your·feed·URI:
576 ······scrapy·crawl·<spider_name>·-o·"%(spider_name)s.jsonl"576 ······scrapy·crawl·<spider_name>·-o·"%(spider_name)s.jsonl"
  
577 Previous·Next577 Previous·Next
578 ===============================================================================578 ===============================================================================
579 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.579 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
580 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.580 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.54 KB
./usr/share/doc/python-scrapy-doc/html/topics/item-pipeline.html
    
Offset 418, 15 lines modifiedOffset 418, 15 lines modified
418 ········<a·href="shell.html"·class="btn·btn-neutral·float-left"·title="Scrapy·shell"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>418 ········<a·href="shell.html"·class="btn·btn-neutral·float-left"·title="Scrapy·shell"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
419 ········<a·href="feed-exports.html"·class="btn·btn-neutral·float-right"·title="Feed·exports"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>419 ········<a·href="feed-exports.html"·class="btn·btn-neutral·float-right"·title="Feed·exports"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
420 ····</div>420 ····</div>
  
421 ··<hr/>421 ··<hr/>
  
422 ··<div·role="contentinfo">422 ··<div·role="contentinfo">
423 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.423 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
424 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.424 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
425 ······</span></p>425 ······</span></p>
426 ··</div>426 ··</div>
  
427 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a427 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
428 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>428 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
429 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.429 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
577 B
html2text {}
    
Offset 274, 9 lines modifiedOffset 274, 9 lines modified
274 }274 }
275 The·integer·values·you·assign·to·classes·in·this·setting·determine·the·order·in275 The·integer·values·you·assign·to·classes·in·this·setting·determine·the·order·in
276 which·they·run:·items·go·through·from·lower·valued·to·higher·valued·classes.276 which·they·run:·items·go·through·from·lower·valued·to·higher·valued·classes.
277 It’s·customary·to·define·these·numbers·in·the·0-1000·range.277 It’s·customary·to·define·these·numbers·in·the·0-1000·range.
  
278 Previous·Next278 Previous·Next
279 ===============================================================================279 ===============================================================================
280 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.280 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
281 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.281 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.47 KB
./usr/share/doc/python-scrapy-doc/html/topics/items.html
    
Offset 555, 15 lines modifiedOffset 555, 15 lines modified
555 ········<a·href="selectors.html"·class="btn·btn-neutral·float-left"·title="Selectors"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>555 ········<a·href="selectors.html"·class="btn·btn-neutral·float-left"·title="Selectors"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
556 ········<a·href="loaders.html"·class="btn·btn-neutral·float-right"·title="Item·Loaders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>556 ········<a·href="loaders.html"·class="btn·btn-neutral·float-right"·title="Item·Loaders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
557 ····</div>557 ····</div>
  
558 ··<hr/>558 ··<hr/>
  
559 ··<div·role="contentinfo">559 ··<div·role="contentinfo">
560 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.560 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
561 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.561 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
562 ······</span></p>562 ······</span></p>
563 ··</div>563 ··</div>
  
564 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a564 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
565 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>565 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
566 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.566 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
522 B
html2text {}
    
Offset 328, 9 lines modifiedOffset 328, 9 lines modified
  
328 *****·Other·classes·related·to·items¶·*****328 *****·Other·classes·related·to·items¶·*****
329 ··classscrapy.item.ItemMeta(class_name,·bases,·attrs)[source]¶329 ··classscrapy.item.ItemMeta(class_name,·bases,·attrs)[source]¶
330 ······Metaclass·of·Item·that·handles·field·definitions.330 ······Metaclass·of·Item·that·handles·field·definitions.
  
331 Previous·Next331 Previous·Next
332 ===============================================================================332 ===============================================================================
333 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.333 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
334 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.334 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/jobs.html
    
Offset 222, 15 lines modifiedOffset 222, 15 lines modified
222 ········<a·href="benchmarking.html"·class="btn·btn-neutral·float-left"·title="Benchmarking"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>222 ········<a·href="benchmarking.html"·class="btn·btn-neutral·float-left"·title="Benchmarking"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
223 ········<a·href="coroutines.html"·class="btn·btn-neutral·float-right"·title="Coroutines"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>223 ········<a·href="coroutines.html"·class="btn·btn-neutral·float-right"·title="Coroutines"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
224 ····</div>224 ····</div>
  
225 ··<hr/>225 ··<hr/>
  
226 ··<div·role="contentinfo">226 ··<div·role="contentinfo">
227 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.227 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
228 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.228 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
229 ······</span></p>229 ······</span></p>
230 ··</div>230 ··</div>
  
231 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a231 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
232 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>232 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
233 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.233 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
576 B
html2text {}
    
Offset 121, 9 lines modifiedOffset 121, 9 lines modified
121 which·must·be·methods·of·the·running·Spider·class.121 which·must·be·methods·of·the·running·Spider·class.
122 If·you·wish·to·log·the·requests·that·couldn’t·be·serialized,·you·can·set·the122 If·you·wish·to·log·the·requests·that·couldn’t·be·serialized,·you·can·set·the
123 SCHEDULER_DEBUG·setting·to·True·in·the·project’s·settings·page.·It·is·False123 SCHEDULER_DEBUG·setting·to·True·in·the·project’s·settings·page.·It·is·False
124 by·default.124 by·default.
  
125 Previous·Next125 Previous·Next
126 ===============================================================================126 ===============================================================================
127 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.127 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
128 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.128 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.59 KB
./usr/share/doc/python-scrapy-doc/html/topics/leaks.html
    
Offset 415, 15 lines modifiedOffset 415, 15 lines modified
415 ········<a·href="dynamic-content.html"·class="btn·btn-neutral·float-left"·title="Selecting·dynamically-loaded·content"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>415 ········<a·href="dynamic-content.html"·class="btn·btn-neutral·float-left"·title="Selecting·dynamically-loaded·content"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
416 ········<a·href="media-pipeline.html"·class="btn·btn-neutral·float-right"·title="Downloading·and·processing·files·and·images"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>416 ········<a·href="media-pipeline.html"·class="btn·btn-neutral·float-right"·title="Downloading·and·processing·files·and·images"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
417 ····</div>417 ····</div>
  
418 ··<hr/>418 ··<hr/>
  
419 ··<div·role="contentinfo">419 ··<div·role="contentinfo">
420 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.420 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
421 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.421 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
422 ······</span></p>422 ······</span></p>
423 ··</div>423 ··</div>
  
424 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a424 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
425 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>425 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
426 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.426 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
574 B
html2text {}
    
Offset 269, 9 lines modifiedOffset 269, 9 lines modified
269 ·····require·significant·changes·to·the·Python·interpreter.269 ·····require·significant·changes·to·the·Python·interpreter.
270 To·keep·memory·consumption·reasonable·you·can·split·the·job·into·several270 To·keep·memory·consumption·reasonable·you·can·split·the·job·into·several
271 smaller·jobs·or·enable·persistent_job_queue·and·stop/start·spider·from·time·to271 smaller·jobs·or·enable·persistent_job_queue·and·stop/start·spider·from·time·to
272 time.272 time.
  
273 Previous·Next273 Previous·Next
274 ===============================================================================274 ===============================================================================
275 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.275 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
276 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.276 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.61 KB
./usr/share/doc/python-scrapy-doc/html/topics/link-extractors.html
    
Offset 304, 15 lines modifiedOffset 304, 15 lines modified
304 ········<a·href="request-response.html"·class="btn·btn-neutral·float-left"·title="Requests·and·Responses"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>304 ········<a·href="request-response.html"·class="btn·btn-neutral·float-left"·title="Requests·and·Responses"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
305 ········<a·href="settings.html"·class="btn·btn-neutral·float-right"·title="Settings"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>305 ········<a·href="settings.html"·class="btn·btn-neutral·float-right"·title="Settings"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
306 ····</div>306 ····</div>
  
307 ··<hr/>307 ··<hr/>
  
308 ··<div·role="contentinfo">308 ··<div·role="contentinfo">
309 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.309 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
310 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.310 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
311 ······</span></p>311 ······</span></p>
312 ··</div>312 ··</div>
  
313 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a313 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
314 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>314 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
315 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.315 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
628 B
html2text {}
    
Offset 114, 9 lines modifiedOffset 114, 9 lines modified
114 ················*·fragment·â€“·the·part·of·the·url·after·the·hash·symbol.·From114 ················*·fragment·â€“·the·part·of·the·url·after·the·hash·symbol.·From
115 ··················the·sample,·this·is·foo.115 ··················the·sample,·this·is·foo.
116 ················*·nofollow·â€“·an·indication·of·the·presence·or·absence·of·a116 ················*·nofollow·â€“·an·indication·of·the·presence·or·absence·of·a
117 ··················nofollow·value·in·the·rel·attribute·of·the·anchor·tag.117 ··················nofollow·value·in·the·rel·attribute·of·the·anchor·tag.
  
118 Previous·Next118 Previous·Next
119 ===============================================================================119 ===============================================================================
120 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.120 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
121 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.121 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.54 KB
./usr/share/doc/python-scrapy-doc/html/topics/loaders.html
    
Offset 815, 15 lines modifiedOffset 815, 15 lines modified
815 ········<a·href="items.html"·class="btn·btn-neutral·float-left"·title="Items"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>815 ········<a·href="items.html"·class="btn·btn-neutral·float-left"·title="Items"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
816 ········<a·href="shell.html"·class="btn·btn-neutral·float-right"·title="Scrapy·shell"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>816 ········<a·href="shell.html"·class="btn·btn-neutral·float-right"·title="Scrapy·shell"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
817 ····</div>817 ····</div>
  
818 ··<hr/>818 ··<hr/>
  
819 ··<div·role="contentinfo">819 ··<div·role="contentinfo">
820 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.820 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
821 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.821 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
822 ······</span></p>822 ······</span></p>
823 ··</div>823 ··</div>
  
824 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a824 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
825 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>825 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
826 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.826 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
601 B
html2text {}
    
Offset 544, 9 lines modifiedOffset 544, 9 lines modified
544 Loaders,·and·different·Item·Loaders·hierarchies·may·fit·better·for·different544 Loaders,·and·different·Item·Loaders·hierarchies·may·fit·better·for·different
545 projects.·Scrapy·only·provides·the·mechanism;·it·doesn’t·impose·any·specific545 projects.·Scrapy·only·provides·the·mechanism;·it·doesn’t·impose·any·specific
546 organization·of·your·Loaders·collection·-·that’s·up·to·you·and·your546 organization·of·your·Loaders·collection·-·that’s·up·to·you·and·your
547 project’s·needs.547 project’s·needs.
  
548 Previous·Next548 Previous·Next
549 ===============================================================================549 ===============================================================================
550 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.550 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
551 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.551 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.44 KB
./usr/share/doc/python-scrapy-doc/html/topics/logging.html
    
Offset 565, 15 lines modifiedOffset 565, 15 lines modified
565 ········<a·href="exceptions.html"·class="btn·btn-neutral·float-left"·title="Exceptions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>565 ········<a·href="exceptions.html"·class="btn·btn-neutral·float-left"·title="Exceptions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
566 ········<a·href="stats.html"·class="btn·btn-neutral·float-right"·title="Stats·Collection"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>566 ········<a·href="stats.html"·class="btn·btn-neutral·float-right"·title="Stats·Collection"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
567 ····</div>567 ····</div>
  
568 ··<hr/>568 ··<hr/>
  
569 ··<div·role="contentinfo">569 ··<div·role="contentinfo">
570 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.570 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
571 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.571 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
572 ······</span></p>572 ······</span></p>
573 ··</div>573 ··</div>
  
574 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a574 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
575 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>575 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
576 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.576 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
485 B
html2text {}
    
Offset 365, 9 lines modifiedOffset 365, 9 lines modified
365 ··········level=logging.INFO365 ··········level=logging.INFO
366 ······)366 ······)
367 ······Refer·to·Run_Scrapy_from_a_script·for·more·details·about·using·Scrapy367 ······Refer·to·Run_Scrapy_from_a_script·for·more·details·about·using·Scrapy
368 ······this·way.368 ······this·way.
  
369 Previous·Next369 Previous·Next
370 ===============================================================================370 ===============================================================================
371 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.371 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
372 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.372 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.47 KB
./usr/share/doc/python-scrapy-doc/html/topics/media-pipeline.html
    
Offset 794, 15 lines modifiedOffset 794, 15 lines modified
794 ········<a·href="leaks.html"·class="btn·btn-neutral·float-left"·title="Debugging·memory·leaks"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>794 ········<a·href="leaks.html"·class="btn·btn-neutral·float-left"·title="Debugging·memory·leaks"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
795 ········<a·href="deploy.html"·class="btn·btn-neutral·float-right"·title="Deploying·Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>795 ········<a·href="deploy.html"·class="btn·btn-neutral·float-right"·title="Deploying·Spiders"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
796 ····</div>796 ····</div>
  
797 ··<hr/>797 ··<hr/>
  
798 ··<div·role="contentinfo">798 ··<div·role="contentinfo">
799 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.799 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
800 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.800 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
801 ······</span></p>801 ······</span></p>
802 ··</div>802 ··</div>
  
803 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a803 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
804 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>804 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
805 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.805 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
492 B
html2text {}
    
Offset 539, 9 lines modifiedOffset 539, 9 lines modified
539 path·to·the·ITEM_PIPELINES·setting,·like·in·the·following·example:539 path·to·the·ITEM_PIPELINES·setting,·like·in·the·following·example:
540 ITEM_PIPELINES·=·{540 ITEM_PIPELINES·=·{
541 ····'myproject.pipelines.MyImagesPipeline':·300541 ····'myproject.pipelines.MyImagesPipeline':·300
542 }542 }
  
543 Previous·Next543 Previous·Next
544 ===============================================================================544 ===============================================================================
545 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.545 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
546 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.546 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.54 KB
./usr/share/doc/python-scrapy-doc/html/topics/practices.html
    
Offset 389, 15 lines modifiedOffset 389, 15 lines modified
389 ········<a·href="contracts.html"·class="btn·btn-neutral·float-left"·title="Spiders·Contracts"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>389 ········<a·href="contracts.html"·class="btn·btn-neutral·float-left"·title="Spiders·Contracts"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
390 ········<a·href="broad-crawls.html"·class="btn·btn-neutral·float-right"·title="Broad·Crawls"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>390 ········<a·href="broad-crawls.html"·class="btn·btn-neutral·float-right"·title="Broad·Crawls"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
391 ····</div>391 ····</div>
  
392 ··<hr/>392 ··<hr/>
  
393 ··<div·role="contentinfo">393 ··<div·role="contentinfo">
394 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.394 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
395 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.395 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
396 ······</span></p>396 ······</span></p>
397 ··</div>397 ··</div>
  
398 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a398 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
399 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>399 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
400 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.400 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
571 B
html2text {}
    
Offset 279, 9 lines modifiedOffset 279, 9 lines modified
279 ······you·can·just·focus·on·parsing·clean·pages.·One·example·of·such279 ······you·can·just·focus·on·parsing·clean·pages.·One·example·of·such
280 ······downloaders·is·Zyte_Smart_Proxy_Manager280 ······downloaders·is·Zyte_Smart_Proxy_Manager
281 If·you·are·still·unable·to·prevent·your·bot·getting·banned,·consider·contacting281 If·you·are·still·unable·to·prevent·your·bot·getting·banned,·consider·contacting
282 commercial_support.282 commercial_support.
  
283 Previous·Next283 Previous·Next
284 ===============================================================================284 ===============================================================================
285 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.285 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
286 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.286 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.58 KB
./usr/share/doc/python-scrapy-doc/html/topics/request-response.html
    
Offset 1662, 15 lines modifiedOffset 1662, 15 lines modified
1662 ········<a·href="feed-exports.html"·class="btn·btn-neutral·float-left"·title="Feed·exports"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1662 ········<a·href="feed-exports.html"·class="btn·btn-neutral·float-left"·title="Feed·exports"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1663 ········<a·href="link-extractors.html"·class="btn·btn-neutral·float-right"·title="Link·Extractors"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1663 ········<a·href="link-extractors.html"·class="btn·btn-neutral·float-right"·title="Link·Extractors"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1664 ····</div>1664 ····</div>
  
1665 ··<hr/>1665 ··<hr/>
  
1666 ··<div·role="contentinfo">1666 ··<div·role="contentinfo">
1667 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1667 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1668 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1668 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1669 ······</span></p>1669 ······</span></p>
1670 ··</div>1670 ··</div>
  
1671 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1671 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1672 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1672 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1673 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1673 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
595 B
html2text {}
    
Offset 1056, 9 lines modifiedOffset 1056, 9 lines modified
1056 ··classscrapy.http.XmlResponse(url[,·...])[source]¶1056 ··classscrapy.http.XmlResponse(url[,·...])[source]¶
1057 ······The·XmlResponse·class·is·a·subclass·of·TextResponse·which·adds·encoding1057 ······The·XmlResponse·class·is·a·subclass·of·TextResponse·which·adds·encoding
1058 ······auto-discovering·support·by·looking·into·the·XML·declaration·line.·See1058 ······auto-discovering·support·by·looking·into·the·XML·declaration·line.·See
1059 ······TextResponse.encoding.1059 ······TextResponse.encoding.
  
1060 Previous·Next1060 Previous·Next
1061 ===============================================================================1061 ===============================================================================
1062 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.1062 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
1063 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.1063 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.53 KB
./usr/share/doc/python-scrapy-doc/html/topics/scheduler.html
    
Offset 358, 15 lines modifiedOffset 358, 15 lines modified
358 ········<a·href="signals.html"·class="btn·btn-neutral·float-left"·title="Signals"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>358 ········<a·href="signals.html"·class="btn·btn-neutral·float-left"·title="Signals"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
359 ········<a·href="exporters.html"·class="btn·btn-neutral·float-right"·title="Item·Exporters"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>359 ········<a·href="exporters.html"·class="btn·btn-neutral·float-right"·title="Item·Exporters"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
360 ····</div>360 ····</div>
  
361 ··<hr/>361 ··<hr/>
  
362 ··<div·role="contentinfo">362 ··<div·role="contentinfo">
363 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.363 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
364 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.364 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
365 ······</span></p>365 ······</span></p>
366 ··</div>366 ··</div>
  
367 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a367 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
368 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>368 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
369 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.369 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
579 B
html2text {}
    
Offset 213, 9 lines modifiedOffset 213, 9 lines modified
213 ···············1.·initialize·the·memory·queue213 ···············1.·initialize·the·memory·queue
214 ···············2.·initialize·the·disk·queue·if·the·jobdir·attribute·is·a·valid214 ···············2.·initialize·the·disk·queue·if·the·jobdir·attribute·is·a·valid
215 ··················directory215 ··················directory
216 ···············3.·return·the·result·of·the·dupefilter’s·open·method216 ···············3.·return·the·result·of·the·dupefilter’s·open·method
  
217 Previous·Next217 Previous·Next
218 ===============================================================================218 ===============================================================================
219 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.219 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
220 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.220 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.2 KB
./usr/share/doc/python-scrapy-doc/html/topics/scrapyd.html
    
Offset 145, 15 lines modifiedOffset 145, 15 lines modified
145 ···········</div>145 ···········</div>
146 ··········</div>146 ··········</div>
147 ··········<footer>147 ··········<footer>
  
148 ··<hr/>148 ··<hr/>
  
149 ··<div·role="contentinfo">149 ··<div·role="contentinfo">
150 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.150 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
151 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.151 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
152 ······</span></p>152 ······</span></p>
153 ··</div>153 ··</div>
  
154 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a154 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
155 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>155 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
156 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.156 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
575 B
html2text {}
    
Offset 65, 9 lines modifiedOffset 65, 9 lines modified
65 ===============================================================================65 ===============================================================================
66 ******·Scrapyd¶·******66 ******·Scrapyd¶·******
67 Scrapyd·has·been·moved·into·a·separate·project.67 Scrapyd·has·been·moved·into·a·separate·project.
68 Its·documentation·is·now·hosted·at:68 Its·documentation·is·now·hosted·at:
69 ·····https://scrapyd.readthedocs.io/en/latest/69 ·····https://scrapyd.readthedocs.io/en/latest/
  
70 ===============================================================================70 ===============================================================================
71 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.71 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
72 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.72 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.5 KB
./usr/share/doc/python-scrapy-doc/html/topics/selectors.html
    
Offset 1329, 15 lines modifiedOffset 1329, 15 lines modified
1329 ········<a·href="spiders.html"·class="btn·btn-neutral·float-left"·title="Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1329 ········<a·href="spiders.html"·class="btn·btn-neutral·float-left"·title="Spiders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1330 ········<a·href="items.html"·class="btn·btn-neutral·float-right"·title="Items"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1330 ········<a·href="items.html"·class="btn·btn-neutral·float-right"·title="Items"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1331 ····</div>1331 ····</div>
  
1332 ··<hr/>1332 ··<hr/>
  
1333 ··<div·role="contentinfo">1333 ··<div·role="contentinfo">
1334 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1334 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1335 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1335 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1336 ······</span></p>1336 ······</span></p>
1337 ··</div>1337 ··</div>
  
1338 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1338 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1339 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1339 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1340 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1340 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
560 B
html2text {}
    
Offset 925, 9 lines modifiedOffset 925, 9 lines modified
925 ···2.·Extract·all·prices·from·a·Google_Base_XML_feed·which·requires·registering925 ···2.·Extract·all·prices·from·a·Google_Base_XML_feed·which·requires·registering
926 ······a·namespace:926 ······a·namespace:
927 ······sel.register_namespace("g",·"http://base.google.com/ns/1.0")927 ······sel.register_namespace("g",·"http://base.google.com/ns/1.0")
928 ······sel.xpath("//g:price").getall()928 ······sel.xpath("//g:price").getall()
  
929 Previous·Next929 Previous·Next
930 ===============================================================================930 ===============================================================================
931 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.931 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
932 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.932 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.44 KB
./usr/share/doc/python-scrapy-doc/html/topics/settings.html
    
Offset 1577, 15 lines modifiedOffset 1577, 15 lines modified
1577 ········<a·href="link-extractors.html"·class="btn·btn-neutral·float-left"·title="Link·Extractors"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1577 ········<a·href="link-extractors.html"·class="btn·btn-neutral·float-left"·title="Link·Extractors"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1578 ········<a·href="exceptions.html"·class="btn·btn-neutral·float-right"·title="Exceptions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1578 ········<a·href="exceptions.html"·class="btn·btn-neutral·float-right"·title="Exceptions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1579 ····</div>1579 ····</div>
  
1580 ··<hr/>1580 ··<hr/>
  
1581 ··<div·role="contentinfo">1581 ··<div·role="contentinfo">
1582 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1582 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1583 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1583 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1584 ······</span></p>1584 ······</span></p>
1585 ··</div>1585 ··</div>
  
1586 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1586 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1587 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1587 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1588 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1588 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
466 B
html2text {}
    
Offset 1227, 9 lines modifiedOffset 1227, 9 lines modified
1227 ····*·TELNETCONSOLE_HOST1227 ····*·TELNETCONSOLE_HOST
1228 ····*·TELNETCONSOLE_PASSWORD1228 ····*·TELNETCONSOLE_PASSWORD
1229 ····*·TELNETCONSOLE_PORT1229 ····*·TELNETCONSOLE_PORT
1230 ····*·TELNETCONSOLE_USERNAME1230 ····*·TELNETCONSOLE_USERNAME
  
1231 Previous·Next1231 Previous·Next
1232 ===============================================================================1232 ===============================================================================
1233 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.1233 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
1234 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.1234 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.52 KB
./usr/share/doc/python-scrapy-doc/html/topics/shell.html
    
Offset 424, 15 lines modifiedOffset 424, 15 lines modified
424 ········<a·href="loaders.html"·class="btn·btn-neutral·float-left"·title="Item·Loaders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>424 ········<a·href="loaders.html"·class="btn·btn-neutral·float-left"·title="Item·Loaders"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
425 ········<a·href="item-pipeline.html"·class="btn·btn-neutral·float-right"·title="Item·Pipeline"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>425 ········<a·href="item-pipeline.html"·class="btn·btn-neutral·float-right"·title="Item·Pipeline"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
426 ····</div>426 ····</div>
  
427 ··<hr/>427 ··<hr/>
  
428 ··<div·role="contentinfo">428 ··<div·role="contentinfo">
429 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.429 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
430 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.430 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
431 ······</span></p>431 ······</span></p>
432 ··</div>432 ··</div>
  
433 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a433 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
434 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>434 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
435 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.435 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
563 B
html2text {}
    
Offset 284, 9 lines modifiedOffset 284, 9 lines modified
284 ...284 ...
285 Note·that·you·can’t·use·the·fetch·shortcut·here·since·the·Scrapy·engine·is285 Note·that·you·can’t·use·the·fetch·shortcut·here·since·the·Scrapy·engine·is
286 blocked·by·the·shell.·However,·after·you·leave·the·shell,·the·spider·will286 blocked·by·the·shell.·However,·after·you·leave·the·shell,·the·spider·will
287 continue·crawling·where·it·stopped,·as·shown·above.287 continue·crawling·where·it·stopped,·as·shown·above.
  
288 Previous·Next288 Previous·Next
289 ===============================================================================289 ===============================================================================
290 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.290 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
291 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.291 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.52 KB
./usr/share/doc/python-scrapy-doc/html/topics/signals.html
    
Offset 663, 15 lines modifiedOffset 663, 15 lines modified
663 ········<a·href="extensions.html"·class="btn·btn-neutral·float-left"·title="Extensions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>663 ········<a·href="extensions.html"·class="btn·btn-neutral·float-left"·title="Extensions"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
664 ········<a·href="scheduler.html"·class="btn·btn-neutral·float-right"·title="Scheduler"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>664 ········<a·href="scheduler.html"·class="btn·btn-neutral·float-right"·title="Scheduler"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
665 ····</div>665 ····</div>
  
666 ··<hr/>666 ··<hr/>
  
667 ··<div·role="contentinfo">667 ··<div·role="contentinfo">
668 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.668 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
669 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.669 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
670 ······</span></p>670 ······</span></p>
671 ··</div>671 ··</div>
  
672 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a672 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
673 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>673 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
674 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.674 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
570 B
html2text {}
    
Offset 404, 9 lines modifiedOffset 404, 9 lines modified
404 ················*·request·(Request·object)·â€“·the·request·that·generated·the404 ················*·request·(Request·object)·â€“·the·request·that·generated·the
405 ··················response405 ··················response
406 ················*·spider·(Spider·object)·â€“·the·spider·for·which·the·response406 ················*·spider·(Spider·object)·â€“·the·spider·for·which·the·response
407 ··················is·intended407 ··················is·intended
  
408 Previous·Next408 Previous·Next
409 ===============================================================================409 ===============================================================================
410 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.410 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
411 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.411 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.59 KB
./usr/share/doc/python-scrapy-doc/html/topics/spider-middleware.html
    
Offset 704, 15 lines modifiedOffset 704, 15 lines modified
704 ········<a·href="downloader-middleware.html"·class="btn·btn-neutral·float-left"·title="Downloader·Middleware"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>704 ········<a·href="downloader-middleware.html"·class="btn·btn-neutral·float-left"·title="Downloader·Middleware"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
705 ········<a·href="extensions.html"·class="btn·btn-neutral·float-right"·title="Extensions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>705 ········<a·href="extensions.html"·class="btn·btn-neutral·float-right"·title="Extensions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
706 ····</div>706 ····</div>
  
707 ··<hr/>707 ··<hr/>
  
708 ··<div·role="contentinfo">708 ··<div·role="contentinfo">
709 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.709 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
710 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.710 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
711 ······</span></p>711 ······</span></p>
712 ··</div>712 ··</div>
  
713 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a713 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
714 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>714 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
715 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.715 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
591 B
html2text {}
    
Offset 435, 9 lines modifiedOffset 435, 9 lines modified
435 ······The·UrlLengthMiddleware·can·be·configured·through·the·following·settings435 ······The·UrlLengthMiddleware·can·be·configured·through·the·following·settings
436 ······(see·the·settings·documentation·for·more·info):436 ······(see·the·settings·documentation·for·more·info):
437 ···············*·URLLENGTH_LIMIT·-·The·maximum·URL·length·to·allow·for437 ···············*·URLLENGTH_LIMIT·-·The·maximum·URL·length·to·allow·for
438 ·················crawled·URLs.438 ·················crawled·URLs.
  
439 Previous·Next439 Previous·Next
440 ===============================================================================440 ===============================================================================
441 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.441 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
442 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.442 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.44 KB
./usr/share/doc/python-scrapy-doc/html/topics/spiders.html
    
Offset 1047, 15 lines modifiedOffset 1047, 15 lines modified
1047 ········<a·href="commands.html"·class="btn·btn-neutral·float-left"·title="Command·line·tool"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>1047 ········<a·href="commands.html"·class="btn·btn-neutral·float-left"·title="Command·line·tool"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
1048 ········<a·href="selectors.html"·class="btn·btn-neutral·float-right"·title="Selectors"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>1048 ········<a·href="selectors.html"·class="btn·btn-neutral·float-right"·title="Selectors"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
1049 ····</div>1049 ····</div>
  
1050 ··<hr/>1050 ··<hr/>
  
1051 ··<div·role="contentinfo">1051 ··<div·role="contentinfo">
1052 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.1052 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
1053 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.1053 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
1054 ······</span></p>1054 ······</span></p>
1055 ··</div>1055 ··</div>
  
1056 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a1056 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
1057 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>1057 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
1058 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.1058 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
475 B
html2text {}
    
Offset 730, 9 lines modifiedOffset 730, 9 lines modified
730 ········pass·#·...·scrape·shop·here·...730 ········pass·#·...·scrape·shop·here·...
  
731 ····def·parse_other(self,·response):731 ····def·parse_other(self,·response):
732 ········pass·#·...·scrape·other·here·...732 ········pass·#·...·scrape·other·here·...
  
733 Previous·Next733 Previous·Next
734 ===============================================================================734 ===============================================================================
735 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.735 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
736 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.736 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.58 KB
./usr/share/doc/python-scrapy-doc/html/topics/stats.html
    
Offset 256, 15 lines modifiedOffset 256, 15 lines modified
256 ········<a·href="logging.html"·class="btn·btn-neutral·float-left"·title="Logging"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>256 ········<a·href="logging.html"·class="btn·btn-neutral·float-left"·title="Logging"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
257 ········<a·href="email.html"·class="btn·btn-neutral·float-right"·title="Sending·e-mail"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>257 ········<a·href="email.html"·class="btn·btn-neutral·float-right"·title="Sending·e-mail"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
258 ····</div>258 ····</div>
  
259 ··<hr/>259 ··<hr/>
  
260 ··<div·role="contentinfo">260 ··<div·role="contentinfo">
261 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.261 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
262 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.262 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
263 ······</span></p>263 ······</span></p>
264 ··</div>264 ··</div>
  
265 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a265 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
266 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>266 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
267 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.267 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
641 B
html2text {}
    
Offset 137, 9 lines modifiedOffset 137, 9 lines modified
137 ······does·nothing).·This·stats·collector·can·be·set·via·the·STATS_CLASS137 ······does·nothing).·This·stats·collector·can·be·set·via·the·STATS_CLASS
138 ······setting,·to·disable·stats·collect·in·order·to·improve·performance.138 ······setting,·to·disable·stats·collect·in·order·to·improve·performance.
139 ······However,·the·performance·penalty·of·stats·collection·is·usually·marginal139 ······However,·the·performance·penalty·of·stats·collection·is·usually·marginal
140 ······compared·to·other·Scrapy·workload·like·parsing·pages.140 ······compared·to·other·Scrapy·workload·like·parsing·pages.
  
141 Previous·Next141 Previous·Next
142 ===============================================================================142 ===============================================================================
143 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.143 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
144 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.144 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.47 KB
./usr/share/doc/python-scrapy-doc/html/topics/telnetconsole.html
    
Offset 354, 15 lines modifiedOffset 354, 15 lines modified
354 ········<a·href="email.html"·class="btn·btn-neutral·float-left"·title="Sending·e-mail"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>354 ········<a·href="email.html"·class="btn·btn-neutral·float-left"·title="Sending·e-mail"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
355 ········<a·href="../faq.html"·class="btn·btn-neutral·float-right"·title="Frequently·Asked·Questions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>355 ········<a·href="../faq.html"·class="btn·btn-neutral·float-right"·title="Frequently·Asked·Questions"·accesskey="n"·rel="next">Next·<span·class="fa·fa-arrow-circle-right"·aria-hidden="true"></span></a>
356 ····</div>356 ····</div>
  
357 ··<hr/>357 ··<hr/>
  
358 ··<div·role="contentinfo">358 ··<div·role="contentinfo">
359 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.359 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
360 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.360 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
361 ······</span></p>361 ······</span></p>
362 ··</div>362 ··</div>
  
363 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a363 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
364 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>364 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
365 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.365 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
493 B
html2text {}
    
Offset 198, 9 lines modifiedOffset 198, 9 lines modified
198 ****·TELNETCONSOLE_PASSWORD¶·****198 ****·TELNETCONSOLE_PASSWORD¶·****
199 Default:·None199 Default:·None
200 The·password·used·for·the·telnet·console,·default·behaviour·is·to·have·it200 The·password·used·for·the·telnet·console,·default·behaviour·is·to·have·it
201 autogenerated201 autogenerated
  
202 Previous·Next202 Previous·Next
203 ===============================================================================203 ===============================================================================
204 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.204 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
205 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.205 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
⊟
1.4 KB
./usr/share/doc/python-scrapy-doc/html/versioning.html
    
Offset 195, 15 lines modifiedOffset 195, 15 lines modified
195 ··········<footer><div·class="rst-footer-buttons"·role="navigation"·aria-label="Footer">195 ··········<footer><div·class="rst-footer-buttons"·role="navigation"·aria-label="Footer">
196 ········<a·href="contributing.html"·class="btn·btn-neutral·float-left"·title="Contributing·to·Scrapy"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>196 ········<a·href="contributing.html"·class="btn·btn-neutral·float-left"·title="Contributing·to·Scrapy"·accesskey="p"·rel="prev"><span·class="fa·fa-arrow-circle-left"·aria-hidden="true"></span>·Previous</a>
197 ····</div>197 ····</div>
  
198 ··<hr/>198 ··<hr/>
  
199 ··<div·role="contentinfo">199 ··<div·role="contentinfo">
200 ····<p>&#169;·Copyright·2008–2024,·Scrapy·developers.200 ····<p>&#169;·Copyright·2008–2025,·Scrapy·developers.
201 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.201 ······<span·class="lastupdated">Last·updated·on·Mar·26,·2023.
202 ······</span></p>202 ······</span></p>
203 ··</div>203 ··</div>
  
204 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a204 ··Built·with·<a·href="https://www.sphinx-doc.org/">Sphinx</a>·using·a
205 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>205 ····<a·href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
206 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.206 ····provided·by·<a·href="https://readthedocs.org">Read·the·Docs</a>.
⊟
534 B
html2text {}
    
Offset 104, 9 lines modifiedOffset 104, 9 lines modified
104 Any·new·Scrapy·release·after·a·year·may·remove·support·for·that·deprecated104 Any·new·Scrapy·release·after·a·year·may·remove·support·for·that·deprecated
105 feature.105 feature.
106 All·deprecated·features·removed·in·a·Scrapy·release·are·explicitly·mentioned·in106 All·deprecated·features·removed·in·a·Scrapy·release·are·explicitly·mentioned·in
107 the·release_notes.107 the·release_notes.
  
108 Previous108 Previous
109 ===============================================================================109 ===============================================================================
110 ©·Copyright·2008–2024,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.110 ©·Copyright·2008–2025,·Scrapy·developers.·Last·updated·on·Mar·26,·2023.
111 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.111 Built·with·Sphinx·using·a·theme·provided·by·Read_the_Docs.
Generated by diffoscope 240