$ python3 s3-bucket-find-direct-url.py https://files.smeet.com Fetching https://files.smeet.com 200 https://files.smeet.com Fetching https://s3.amazonaws.com/files.smeet.com/ 200 https://s3.amazonaws.com/files.smeet.com/ Found the bucket: https://files.smeet.com == https://s3.amazonaws.com/files.smeet.com/ Fetching https://files.smeet.com.s3.amazonaws.com/ 200 https://files.smeet.com.s3.amazonaws.com/ Found the bucket: https://files.smeet.com == https://files.smeet.com.s3.amazonaws.com/ Fetching https://storage.googleapis.com/files.smeet.com/ 404 https://storage.googleapis.com/files.smeet.com/ Fetching https://s3.nl-ams.scw.cloud/files.smeet.com/ 404 https://s3.nl-ams.scw.cloud/files.smeet.com/ Fetching https://s3.fr-par.scw.cloud/files.smeet.com/ 404 https://s3.fr-par.scw.cloud/files.smeet.com/ Fetching https://s3.wasabisys.com/files.smeet.com/ 404 https://s3.wasabisys.com/files.smeet.com/ $ python3 s3-bucket-list.py --with-list-urls https://files.smeet.com.s3.amazonaws.com > files.smeet.com.s3.amazonaws.com-urls.txt 3> files.smeet.com.s3.amazonaws.com-marker-urls.txt Traceback (most recent call last): File "s3-bucket-list.py", line 76, in resp = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 320, in begin version, status, reason = self._read_status() File "/usr/lib/python3.6/http/client.py", line 289, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response $ python3 s3-bucket-list.py --with-list-urls https://s3.amazonaws.com/files.smeet.com/ > s3.amazonaws.com-files.smeet.com-urls.txt 3> s3.amazonaws.com-files.smeet.com-marker-urls.txt Traceback (most recent call last): File "s3-bucket-list.py", line 76, in resp = conn.getresponse() File "/usr/lib/python3.6/http/client.py", line 1377, in getresponse response.begin() File "/usr/lib/python3.6/http/client.py", line 320, in begin version, status, reason = self._read_status() File "/usr/lib/python3.6/http/client.py", line 289, in _read_status raise RemoteDisconnected("Remote end closed connection without" http.client.RemoteDisconnected: Remote end closed connection without response $ wc -l files.smeet.com.s3.amazonaws.com-* s3.amazonaws.com-files.smeet.com-* 16 files.smeet.com.s3.amazonaws.com-marker-urls.txt 15000 files.smeet.com.s3.amazonaws.com-urls.txt 16 s3.amazonaws.com-files.smeet.com-marker-urls.txt 15000 s3.amazonaws.com-files.smeet.com-urls.txt 30032 total