提取中断,每次不同地方

来源:2-3 整合爬虫功能函数

weixin_精慕门2533000

2021-09-02 16:04:19

Traceback (most recent call last):

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 438, in _error_catcher

    yield

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 767, in read_chunked

    chunk = self._handle_chunk(amt)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 715, in _handle_chunk

    value = self._fp._safe_read(amt)

  File "C:\Users\hongli.fan\Anaconda3\lib\http\client.py", line 614, in _safe_read

    raise IncompleteRead(data, amt-len(data))

http.client.IncompleteRead: IncompleteRead(5405 bytes read, 4835 more expected)


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\models.py", line 753, in generate

    for chunk in self.raw.stream(chunk_size, decode_content=True):

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 572, in stream

    for line in self.read_chunked(amt, decode_content=decode_content):

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 793, in read_chunked

    self._original_response.close()

  File "C:\Users\hongli.fan\Anaconda3\lib\contextlib.py", line 131, in __exit__

    self.gen.throw(type, value, traceback)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\urllib3\response.py", line 455, in _error_catcher

    raise ProtocolError("Connection broken: %r" % e, e)

urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(5405 bytes read, 4835 more expected)', IncompleteRead(5405 bytes read, 4835 more expected))


During handling of the above exception, another exception occurred:


Traceback (most recent call last):

  File "D:/新建文件夹/dat_learn/7爬虫/7.7整合爬虫函数.py", line 67, in <module>

    print(get_infor(lis))

  File "D:/新建文件夹/dat_learn/7爬虫/7.7整合爬虫函数.py", line 37, in get_infor

    r = requests.get(url=url, headers=headers)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\api.py", line 76, in get

    return request('get', url, params=params, **kwargs)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\api.py", line 61, in request

    return session.request(method=method, url=url, **kwargs)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\sessions.py", line 542, in request

    resp = self.send(prep, **send_kwargs)

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\sessions.py", line 697, in send

    r.content

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\models.py", line 831, in content

    self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''

  File "C:\Users\hongli.fan\Anaconda3\lib\site-packages\requests\models.py", line 756, in generate

    raise ChunkedEncodingError(e)

requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(5405 bytes read, 4835 more expected)', IncompleteRead(5405 bytes read, 4835 more expected))


提取数据的时候,经常停止的地方不一样,单个网页测试提取是正常的,这是网络问题吗?

写回答

1回答

DeltaF

2021-09-06

截图,给我看一下报错的那一行代码信息,直接看报错最后的信息就好

0

人人都能学的数据分析

从0开始学数据分析,未来互联网各岗位的标配技能,产品经理、运营经理、技术人员人人必备,人人必会的的实用技能

1411 学习 · 1133 问题

查看课程