lain/twitter-scrapper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
331 commits 1 branch 0 tags 447 KiB
Commit graph

331 commits

This branch
This branch
All branches
Author SHA1 Message Date
Alexander Sheiko
391e443058 Use search GraphQL API 
Close #109 #116
 2023-07-03 14:38:46 +03:00
Alexander Sheiko
07ad3789ef Authentication is required! 
(quick dirty fix)
Close #115
 2023-07-02 01:41:48 +03:00
Nomadic
5b0c8c3f9f Merge pull request #112 from helios2003/master 
Fixed typo in README in the section "Search profiles by query".
 2023-06-25 23:50:53 +03:00
Ankit Dash
5032d6517d Update README.md 2023-06-26 02:13:01 +05:30
Nomadic
2c19aab28b Merge pull request #110 from Veetaha/feat/get-tweet-gifs 
Parse GIFs for in the GetTweet API
 2023-06-20 10:47:53 +03:00
Veetaha
9c3764f484 Parse GIFs for in the GetTweet API 
I am writing an app that needs to get info about all media in a tweet and forward it to a Telegram chat.

Today animated GIFs are ignored in the response of TweetDetail, although the are there (exept for the caveat mentioned below). So without this change the GIFs are not present in the twitterscraper.Tweet struct.

Following the analogy with the split between Photos and Videos I added GIFs to the Tweet type.

There is one caveat that I found during testing that I can't really explain. But GIFs don't occur in the response unless the bearerToken2 is set. I don't know what this token means, maybe it somehow identifies a destop-browser variant of twitter frontend, but with this token the GIFs are present in the response.

Please note that I never wrote Go code before in my life. I am using this library via the FFI to link it to my Rust codebase.
 2023-06-18 19:35:34 +02:00
Alexander Sheiko
15e5f233b7 Support 2FA 
Close #106
 2023-06-12 18:23:26 +03:00
Alexander Sheiko
d9d8ee67b6 Support for threaded tweets 
Close #84
 2023-06-05 13:49:56 +03:00
Alexander Sheiko
0f96589c74 Remove skip pinned tweet 
Close #103
 2023-06-04 13:12:21 +03:00
Alexander Sheiko
50440667ed Use GraphQL API with timeline v2 
Close #85
Close #82
Close #77
Close #76
 2023-06-01 23:20:11 +03:00
Alexander Sheiko
c07bd1d1d4 improve request 2023-06-01 23:05:37 +03:00
Alexander Sheiko
e52022e31a Add LoginOpenAccount 
Close #101 again
 2023-05-30 17:31:00 +03:00
Alexander Sheiko
16e8b34638 Fix trends 2023-05-30 17:14:38 +03:00
Alexander Sheiko
10887ac302 Skip test with authentication 2023-05-21 01:29:08 +03:00
Nomadic
0c2489c695 Merge pull request #99 from regynald/master 
Add view count to tweets
 2023-05-21 01:14:33 +03:00
Nomadic
b2a9b3c722 Update search example 
Close #100
 2023-05-21 01:10:22 +03:00
Regynald Augustin
45d30f7304 Add test 2023-05-18 15:51:03 -04:00
Regynald Augustin
b458d2bd82 Add views to timeline and parse 2023-05-18 15:18:29 -04:00
Regynald Augustin
048fce5cf2 Add views to tweet type 2023-05-18 15:18:17 -04:00
Alexander Sheiko
66780eb801 Remove deprecated functions 
Removed global default scraper
 2023-05-11 14:15:56 +03:00
Alexander Sheiko
3eb86161ce Fix 403 error 2023-05-11 13:36:32 +03:00
Alexander Sheiko
848bdfd3cf Bump go version to 1.16 2023-05-10 22:29:40 +03:00
Alexander Sheiko
90f9a71721 Improve logout 
drop session in Twitter
 2023-05-10 22:24:32 +03:00
Alexander Sheiko
e86c2fa4ce Improve login flow 
support LoginAcid task
 2023-05-10 21:48:41 +03:00
Alexander Sheiko
3bb78070b7 Add creds for CI 2023-05-10 17:43:02 +03:00
Nomadic
cd3e612419 Merge pull request #95 from windowsdeveloperwannabe/tweets-by-id 
Fetch tweets by userID
 2023-05-10 17:13:12 +03:00
Nomadic
7c78ad7719 Merge pull request #92 from windowsdeveloperwannabe/user-name 
Add user's full name to tweet
 2023-05-10 17:11:31 +03:00
windowsdeveloperwannabe
76b2215e59 add fetch tweets by userID func 2023-05-10 06:09:11 -07:00
windowsdeveloperwannabe
d7b7a8754a Merge branch 'master' into user-name 2023-05-10 06:05:18 -07:00
Nomadic
e30b24e903 Merge pull request #91 from windowsdeveloperwannabe/mentions 
Add @user mentions to Tweet
 2023-05-10 15:22:10 +03:00
windowsdeveloperwannabe
9010999a81 remove debug 2023-05-10 05:19:27 -07:00
windowsdeveloperwannabe
21d595ff42 add user's full name 2023-05-10 05:05:07 -07:00
windowsdeveloperwannabe
0adefb32f3 add user mentions 2023-05-10 05:02:21 -07:00
Nomadic
5081b46f68 Merge pull request #90 from windowsdeveloperwannabe/photo-id 
Save photo ids similarly to video ids
 2023-05-10 13:51:32 +03:00
Nomadic
0f522145f1 Merge pull request #89 from windowsdeveloperwannabe/patch-1 
Don't replace existing `http.Client`
 2023-05-10 13:50:33 +03:00
windowsdeveloperwannabe
6a929cde9c save photo ids 2023-05-10 03:18:34 -07:00
Alexander Sheiko
f0d7735a76 Add GetCookies and SetCookies 2023-05-10 11:42:47 +03:00
windowsdeveloperwannabe
8a740fe634 don't replace existing client 2023-05-10 08:50:24 +03:00
Alexander Sheiko
deb946cf22 Make IsLoggedIn more active and correct 2023-05-08 13:13:08 +03:00
Alexander Sheiko
ed0a19c7de Fix search again 2023-05-07 17:46:01 +03:00
Alexander Sheiko
61f2b78e2b Fix latest search 
Close #75
 2023-04-30 00:25:35 +03:00
Alexander Sheiko
eb18988829 Add authentication 
Breaking changes: removed WithCookie and WithXCsrfToken
 2023-04-23 17:32:28 +03:00
Alexander Sheiko
3d364abaac Update deps 2023-04-23 17:29:40 +03:00
Nomadic
efb2f50ef4 Merge pull request #75 from l33t7here/master 
Fixed SearchLatest Mode Freeze Issue
 2023-03-23 18:41:30 +02:00
pain
7467c3aba1 Fixed SearchLatest Mode Freeze Issue 2023-03-17 04:11:02 +05:30
Nomadic
ace01ab0b2 Merge pull request #70 from n0madic/dependabot/go_modules/golang.org/x/net-0.7.0 
Bump golang.org/x/net from 0.5.0 to 0.7.0
 2023-03-15 10:41:05 +02:00
Alexander Sheiko
78bf7d6365 Fix cursor test 2023-03-02 11:59:27 +02:00
dependabot[bot]
448d660283 Bump golang.org/x/net from 0.5.0 to 0.7.0 
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0.
- [Release notes](https://github.com/golang/net/releases)
- [Commits](https://github.com/golang/net/compare/v0.5.0...v0.7.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-02-18 03:58:14 +00:00
Alexander Sheiko
7d5b51da84 Fix search 2023-02-16 11:15:13 +02:00
Alexander Sheiko
96325b0173 Up go version for tests 2023-01-12 11:33:43 +02:00