Skip to content

Commit c8f9eea

Browse files
committed
Ver Chg to V1.0.2
1 parent 8945fb8 commit c8f9eea

File tree

7 files changed

+331
-429
lines changed

7 files changed

+331
-429
lines changed

README.md

Lines changed: 37 additions & 75 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
<div align="center">
22

3-
# [TikHub_PyPi](https://pypi.org/project/tikhub)
3+
# [TikHub_API_PyPi](https://pypi.org/project/tikhub)
44

55
[api.tikhub.io](https://api.tikhub.io/docs)」,是一个异步高性能的Douyin,TikTok数据爬取在线工具,此repo为基于该API的封装,方便各位开发者调用。
66

@@ -10,69 +10,22 @@ PyPi:[https://pypi.org/project/tikhub](https://pypi.org/project/tikhub)
1010

1111
</div>
1212

13-
## 注释
1413

15-
> 本项目使用以下Emoji在开发图表中表明开发状态!
16-
17-
| Emoji | 代表的含义 |
18-
| :---: | :---: |
19-
| 🚀 | 火箭 - 功能已编写完成并测试通过,并且已部署至生产环境。|
20-
|| 勾选符 - 功能已编写完成,但还有待测试,测试通过后将部署至生产环境。|
21-
|| 叉号符 - 功能尚未开始编写或还未编写完成。|
22-
| 🔜 | SOON符 - 功能已提出但尚未分配指定开发人员。|
23-
| ⚠️ | 警告符 - 功能出现问题待修复。|
24-
25-
## 项目进度
26-
27-
| 状态 | API端点路径 | 功能 |
28-
| :---: | :---: | :---: |
29-
| 🚀 | `/token` | 生成`Bearer Token` |
30-
| 🚀 | `/users/me/` | 获取用户信息 |
31-
32-
> 各接口端点需求
33-
34-
| 状态 | 支持平台 | 需求 | 开始日期 | ETA日期 | 开发者 |
35-
| :---: | :--- | :---: | :---: | :---: |:---: |
36-
| 🚀 | 抖音, TikTok | 爬取单个视频数据 | 2022/10/08 | 已完成 | @Evil0ctal |
37-
| 🚀 | 抖音, TikTok | 爬取单个视频评论数据 | 2022/10/08 | 已完成 | @Evil0ctal |
38-
| 🚀 | 抖音, TikTok | 爬取配乐作品数据 | 2022/10/08 | 已完成 | @Evil0ctal |
39-
| 🚀 | 抖音, TikTok | 爬取用户主页视频数据 | 2022/10/08 | 已完成 | @Evil0ctal |
40-
| 🚀 | 抖音, TikTok | 爬取用户主页已点赞视频数据 | 2022/10/08 | 已完成 | @Evil0ctal |
41-
42-
> 抖音相关接口生产部署 - API tags: Douyin
43-
44-
| 状态 | API端点路径 | 功能 | issue |
45-
| :---: | :---: | :---: | :---: |
46-
| 🚀 | `/douyin_video_data/` | 爬取单个视频数据 | 无已知问题 |
47-
| ⚠️ | `/douyin_video_comments/` | 爬取单个视频评论数据 | 失效待更新 |
48-
| ⚠️ | `/douyin_music_videos/` | 爬取配乐作品数据 | 失效待更新 |
49-
| 🚀 | `/douyin_profile_videos/` | 爬取用户主页视频数据 | 无已知问题 |
50-
| 🚀 | `/douyin_profile_liked_videos/` | 爬取用户主页已点赞视频数据 | 无已知问题 |
51-
52-
> TikTok相关接口生产部署 - API tags: TikTok
53-
54-
| 状态 | API端点路径 | 功能 | issue |
55-
| :---: | :---: | :---: | :---: |
56-
| 🚀 | `/tiktok_video_data/` | 爬取单个视频数据 | 无已知问题 |
57-
| 🚀 | `/tiktok_video_comments/` | 爬取单个视频评论数据 | 无已知问题 |
58-
| 🚀 | `/tiktok_music_videos/` | 爬取配乐作品数据 | 无已知问题 |
59-
| 🚀 | `/tiktok_profile_videos/` | 爬取用户主页视频数据 | 无已知问题 |
60-
| 🚀 | `/tiktok_profile_liked_videos/` | 爬取用户主页已点赞视频数据 | 无已知问题 |
14+
## 使用示例
6115

62-
## 待办事宜 `Todo` 列表
16+
> 查看[test.py](https://github.com/TikHubIO/TikHub_PyPi/blob/main/test/test.py)
6317
64-
- [ ] ⚠️ 修复`/douyin_video_comments/`端点
65-
- [ ] ⚠️ 修复`/douyin_music_videos/`端点
18+
- Install
6619

67-
## 使用示例
20+
```bash
21+
pip install tikhub
22+
```
6823

69-
> 查看[demo.py](https://github.com/TikHubIO/TikHub_PyPi/blob/main/demo/demo.py)
24+
- Usage
7025

7126
```python
72-
import asyncio
73-
import time
27+
from tikhub.api import *
7428

75-
from tikhub.api import API
7629

7730
async def async_test() -> None:
7831
# 异步测试/Async test
@@ -89,54 +42,62 @@ async def async_test() -> None:
8942
start_time = time.time()
9043

9144
# 获取TikHub请求头/Get TikHub request header
92-
print("Running test : API.authenticate()")
93-
await api.authenticate()
45+
r = await api.user_login()
46+
print("Running test : API.user_login()")
47+
print(r)
9448

9549
# 获取TikHub用户信息/Get TikHub user information
9650
print("Running test : API.get_user_info()")
97-
await api.get_user_info()
51+
r = await api.get_user_info()
52+
print(r)
9853

9954
print("\nRunning ALL TikTok methods test...\n")
10055

10156
# 获取单个视频数据/Get single video data
10257
print("Running test : API.get_tiktok_video_data()")
103-
await api.get_tiktok_video_data(tiktok_url)
58+
r = await api.get_tiktok_video_data(tiktok_url)
59+
# print(r)
10460

10561
# 获取获取用户主页的所有视频数据/Get all video data on the user's homepage
10662
print("Running test : API.get_tiktok_profile_videos()")
107-
aweme_list = await api.get_tiktok_profile_videos(tiktok_url, 20)
108-
print(f'Get {len(aweme_list)} videos from profile')
63+
r = await api.get_tiktok_profile_videos(tiktok_url, cursor=None, count=None, get_all=False)
64+
print(f'Get {len(r)} videos from profile')
10965

11066
# 获取用户主页的所有点赞视频数据/Get all liked video data on the user's homepage
11167
print("Running test : API.get_tiktok_profile_liked_videos()")
112-
aweme_list = await api.get_tiktok_profile_liked_videos(tiktok_url, 20)
113-
print(f'Get {len(aweme_list)} liked videos from profile')
68+
r = await api.get_tiktok_profile_liked_videos(tiktok_url, cursor=None, count=None, get_all=False)
69+
print(f'Get {len(r)} liked videos from profile')
11470

11571
# 获取TikTok视频的所有评论数据/Get all comment data of TikTok video
11672
print("Running test : API.get_tiktok_video_comments()")
117-
comments_list = await api.get_tiktok_video_comments(tiktok_url, 20)
118-
print(f'Get {len(comments_list)} comments from video')
73+
r = await api.get_tiktok_video_comments(tiktok_url, cursor=None, count=None, get_all=False)
74+
print(f'Get {len(r)} comments from video')
11975

12076
# 获取音乐页面上的所有(理论上能抓取到的)视频数据/Get all (theoretically) video data on the music page
12177
print("Running test : API.get_tiktok_music_videos()")
122-
aweme_list = await api.get_tiktok_music_videos(tiktok_music_url, 20)
123-
print(f'Get {len(aweme_list)} videos from music')
78+
r = await api.get_tiktok_music_videos(tiktok_music_url, cursor=None, count=None, get_all=False)
79+
print(f'Get {len(r)} videos from music')
12480

12581
print("\nRunning ALL Douyin methods test...\n")
12682

12783
# 获取单个视频数据/Get single video data
12884
print("Running test : API.get_douyin_video_data()")
129-
await api.get_douyin_video_data(douyin_url)
85+
r = await api.get_douyin_video_data(douyin_url)
13086

13187
# 获取获取用户主页的所有视频数据/Get all video data on the user's homepage
13288
print("Running test : API.get_douyin_profile_videos()")
133-
aweme_list = await api.get_douyin_profile_videos(douyin_user_url, 20)
134-
print(f'Get {len(aweme_list)} videos from profile')
89+
r = await api.get_douyin_profile_videos(douyin_user_url, cursor=None, count=None, get_all=False)
90+
print(f'Get {len(r)} videos from profile')
13591

13692
# 获取用户主页的所有点赞视频数据/Get all liked video data on the user's homepage
13793
print("Running test : API.get_douyin_profile_liked_videos()")
138-
aweme_list = await api.get_douyin_profile_liked_videos(douyin_user_url, 20)
139-
print(f'Get {len(aweme_list)} liked videos from profile')
94+
r = await api.get_douyin_profile_liked_videos(douyin_user_url, cursor=None, count=None, get_all=False)
95+
print(f'Get {len(r)} liked videos from profile')
96+
97+
# 获取抖音视频的所有评论数据/Get all comment data of Douyin video
98+
print("Running test : API.get_douyin_video_comments()")
99+
r = await api.get_douyin_video_comments(douyin_url, cursor=None, count=None, get_all=False)
100+
print(f'Get {len(r)} comments from video')
140101

141102
# 总耗时/Total time
142103
total_time = round(time.time() - start_time, 2)
@@ -145,11 +106,12 @@ async def async_test() -> None:
145106

146107
if __name__ == '__main__':
147108
api = API(
148-
username='test',
149-
password='test',
109+
email='EMAIL@EXAMPLE.COM',
110+
password='PASSWORD',
150111
proxy=None,
151112
)
152113
asyncio.run(async_test())
114+
153115
```
154116

155117

dist/tikhub-1.0.2-py3-none-any.whl

9.71 KB
Binary file not shown.

dist/tikhub-1.0.2.tar.gz

9.88 KB
Binary file not shown.

setup.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@
1414
setuptools.setup(
1515
name='tikhub',
1616
author='TikHub.io',
17-
version='1.0.1',
17+
version='1.0.2',
1818
license='Apache V2.0 License',
1919
description='Douyin/TikTok async data scraper.',
2020
long_description=long_description,
Lines changed: 28 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,4 @@
1-
import asyncio
2-
import time
3-
4-
from tikhub.api import API
1+
from tikhub.api import *
52

63

74
async def async_test() -> None:
@@ -19,54 +16,62 @@ async def async_test() -> None:
1916
start_time = time.time()
2017

2118
# 获取TikHub请求头/Get TikHub request header
22-
print("Running test : API.authenticate()")
23-
await api.authenticate()
19+
r = await api.user_login()
20+
print("Running test : API.user_login()")
21+
print(r)
2422

2523
# 获取TikHub用户信息/Get TikHub user information
2624
print("Running test : API.get_user_info()")
27-
await api.get_user_info()
25+
r = await api.get_user_info()
26+
print(r)
2827

2928
print("\nRunning ALL TikTok methods test...\n")
3029

3130
# 获取单个视频数据/Get single video data
3231
print("Running test : API.get_tiktok_video_data()")
33-
await api.get_tiktok_video_data(tiktok_url)
32+
r = await api.get_tiktok_video_data(tiktok_url)
33+
# print(r)
3434

3535
# 获取获取用户主页的所有视频数据/Get all video data on the user's homepage
3636
print("Running test : API.get_tiktok_profile_videos()")
37-
aweme_list = await api.get_tiktok_profile_videos(tiktok_url, 20)
38-
print(f'Get {len(aweme_list)} videos from profile')
37+
r = await api.get_tiktok_profile_videos(tiktok_url, cursor=None, count=None, get_all=False)
38+
print(f'Get {len(r)} videos from profile')
3939

4040
# 获取用户主页的所有点赞视频数据/Get all liked video data on the user's homepage
4141
print("Running test : API.get_tiktok_profile_liked_videos()")
42-
aweme_list = await api.get_tiktok_profile_liked_videos(tiktok_url, 20)
43-
print(f'Get {len(aweme_list)} liked videos from profile')
42+
r = await api.get_tiktok_profile_liked_videos(tiktok_url, cursor=None, count=None, get_all=False)
43+
print(f'Get {len(r)} liked videos from profile')
4444

4545
# 获取TikTok视频的所有评论数据/Get all comment data of TikTok video
4646
print("Running test : API.get_tiktok_video_comments()")
47-
comments_list = await api.get_tiktok_video_comments(tiktok_url, 20)
48-
print(f'Get {len(comments_list)} comments from video')
47+
r = await api.get_tiktok_video_comments(tiktok_url, cursor=None, count=None, get_all=False)
48+
print(f'Get {len(r)} comments from video')
4949

5050
# 获取音乐页面上的所有(理论上能抓取到的)视频数据/Get all (theoretically) video data on the music page
5151
print("Running test : API.get_tiktok_music_videos()")
52-
aweme_list = await api.get_tiktok_music_videos(tiktok_music_url, 20)
53-
print(f'Get {len(aweme_list)} videos from music')
52+
r = await api.get_tiktok_music_videos(tiktok_music_url, cursor=None, count=None, get_all=False)
53+
print(f'Get {len(r)} videos from music')
5454

5555
print("\nRunning ALL Douyin methods test...\n")
5656

5757
# 获取单个视频数据/Get single video data
5858
print("Running test : API.get_douyin_video_data()")
59-
await api.get_douyin_video_data(douyin_url)
59+
r = await api.get_douyin_video_data(douyin_url)
6060

6161
# 获取获取用户主页的所有视频数据/Get all video data on the user's homepage
6262
print("Running test : API.get_douyin_profile_videos()")
63-
aweme_list = await api.get_douyin_profile_videos(douyin_user_url, 20)
64-
print(f'Get {len(aweme_list)} videos from profile')
63+
r = await api.get_douyin_profile_videos(douyin_user_url, cursor=None, count=None, get_all=False)
64+
print(f'Get {len(r)} videos from profile')
6565

6666
# 获取用户主页的所有点赞视频数据/Get all liked video data on the user's homepage
6767
print("Running test : API.get_douyin_profile_liked_videos()")
68-
aweme_list = await api.get_douyin_profile_liked_videos(douyin_user_url, 20)
69-
print(f'Get {len(aweme_list)} liked videos from profile')
68+
r = await api.get_douyin_profile_liked_videos(douyin_user_url, cursor=None, count=None, get_all=False)
69+
print(f'Get {len(r)} liked videos from profile')
70+
71+
# 获取抖音视频的所有评论数据/Get all comment data of Douyin video
72+
print("Running test : API.get_douyin_video_comments()")
73+
r = await api.get_douyin_video_comments(douyin_url, cursor=None, count=None, get_all=False)
74+
print(f'Get {len(r)} comments from video')
7075

7176
# 总耗时/Total time
7277
total_time = round(time.time() - start_time, 2)
@@ -75,8 +80,8 @@ async def async_test() -> None:
7580

7681
if __name__ == '__main__':
7782
api = API(
78-
username='test',
79-
password='test',
83+
email='EMAIL@EXAMPLE.COM',
84+
password='PASSWORD',
8085
proxy=None,
8186
)
8287
asyncio.run(async_test())

tikhub/aiohttp_client.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
import aiohttp
2+
3+
class AiohttpClient:
4+
_instance = None
5+
6+
def __new__(cls, *args, **kwargs):
7+
if cls._instance is None:
8+
cls._instance = super().__new__(cls)
9+
return cls._instance
10+
11+
async def request(self, method, url, headers=None, params=None, data=None, json=None) -> (int, str, dict):
12+
async with aiohttp.ClientSession() as session:
13+
async with session.request(method, url, headers=headers, params=params, data=data, json=json) as response:
14+
response_text = await response.text()
15+
response_json = None
16+
if response_text:
17+
response_json = await response.json()
18+
return response.status, response_text, response_json

0 commit comments

Comments
 (0)