demo run question

[Maple-geekZhu]

I have already created the environment according to your install.md document, and also created the required folders. Meanwhile, the necessary pth files have been downloaded as well.
I ran the CLI command in the root directory of the project:
python demo/video_demo_with_text.py demo/minions_rush_out.mp4 --out demo_outputs/minions_rush_out_outputs.mp4 --masa_config configs/masa-gdino/masa_gdino_swinb_inference.py --masa_checkpoint saved_models/masa_models/gdino_masa.pth --texts "yellow_minions" --score-thr 0.2 --unified --show_fps
There has been no response for a long time, and no error has been reported either. Why is that? My CUDA is also available, but after running the command, I didn't find my process with nvidia-smi. Did the CUDA call fail or there is other error?
By the way, my system is Ubuntu 22.04 LTS (GNU/Linux 6.2.0-34-generic x86_64). The relevant information about CUDA is NVIDIA-SMI 535.113.01 Driver Version: 535.113.01 CUDA Version: 12.2. And the versions of PyTorch and Python I selected are consistent with those in your install.md.

[wuhuang1313113-cy]

請問這個問題解決了嗎？我也遇到了相同的問題。

[zyhhh123]

same issues

[Maple-geekZhu]

hi, @zyhhh123 @wuhuang1313113-cy I found some bugs and hope it will be helpful to you. After running the demo for a while, an error will be reported. The reason for the error is probably that the script will automatically download the model from Huggingface. Here, it is downloading bert-base-uncased (I'm not sure if there are other packages that need to be downloaded). I debugged the demo script file and found that it was the statement "import masa" that caused the download. I have already downloaded these models, but I don't know where to put them or how to modify the script to use the local models. The following is the error message.
(masaenv) (base) zrt@test:~/masa$ bash run_video_processing.sh
Traceback (most recent call last):
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
raise err
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
sock.connect(sa)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 716, in urlopen
httplib_response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 404, in _make_request
self._validate_conn(conn)
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1061, in _validate_conn
conn.connect()
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connection.py", line 363, in connect
self.sock = conn = self._new_conn()
^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connection.py", line 179, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<urllib3.connection.HTTPSConnection object at 0x7ad01796ff10>, 'Connection to huggingface.co timed out. (connect timeout=10)')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 802, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/urllib3/util/retry.py", line 594, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-uncased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7ad01796ff10>, 'Connection to huggingface.co timed out. (connect timeout=10)'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1376, in _get_metadata_or_catch_error
metadata = get_hf_file_metadata(
^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1296, in get_hf_file_metadata
r = _request_wrapper(
^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 277, in _request_wrapper
response = _request_wrapper(
^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 300, in _request_wrapper
response = get_session().request(method=method, url=url, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/utils/_http.py", line 93, in send
return super().send(request, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/requests/adapters.py", line 553, in send
raise ConnectTimeout(e, request=request)
requests.exceptions.ConnectTimeout: (MaxRetryError("HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /bert-base-uncased/resolve/main/config.json (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7ad01796ff10>, 'Connection to huggingface.co timed out. (connect timeout=10)'))"), '(Request ID: 8099c019-6712-4ad0-be49-7818c5cb8d1e)')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/utils/hub.py", line 398, in cached_file
resolved_file = hf_hub_download(
^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 862, in hf_hub_download
return _hf_hub_download_to_cache_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 969, in _hf_hub_download_to_cache_dir
_raise_on_head_call_error(head_call_error, force_download, local_files_only)
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/huggingface_hub/file_download.py", line 1487, in _raise_on_head_call_error
raise LocalEntryNotFoundError(
huggingface_hub.errors.LocalEntryNotFoundError: An error happened while trying to locate the file on the Hub and we cannot find the requested files in the local cache. Please check your connection and try again or make sure your Internet connection is on.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/data/zrt/masa/demo/video_demo_with_text.py", line 254, in
main()
File "/data/zrt/masa/demo/video_demo_with_text.py", line 96, in main
masa_model = init_masa(args.masa_config, args.masa_checkpoint, device=args.device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/masa/masa/apis/masa_inference.py", line 74, in init_masa
model = MODELS.build(config.model)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
^^^^^^^^^^^^^^^
File "/data/zrt/masa/masa/models/mot/masa.py", line 94, in init
self.detector = MODELS.build(detector)
^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
^^^^^^^^^^^^^^^
File "/data/zrt/masa/masa/models/detectors/gdino_masa.py", line 52, in init
super().init(*args, **kwargs)
File "/data/zrt/masa/masa/models/detectors/grounding_dino.py", line 182, in init
super().init(*args, **kwargs)
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmdet/models/detectors/dino.py", line 30, in init
super().init(*args, **kwargs)
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmdet/models/detectors/deformable_detr.py", line 69, in init
super().init(*args, decoder=decoder, bbox_head=bbox_head, **kwargs)
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmdet/models/detectors/base_detr.py", line 77, in init
self._init_layers()
File "/data/zrt/masa/masa/models/detectors/grounding_dino.py", line 204, in _init_layers
self.language_model = MODELS.build(self.language_model_cfg)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/mmdet/models/language_models/bert.py", line 119, in init
self.tokenizer = AutoTokenizer.from_pretrained(name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/models/auto/tokenization_auto.py", line 782, in from_pretrained
config = AutoConfig.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1111, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/configuration_utils.py", line 633, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/configuration_utils.py", line 688, in _get_config_dict
resolved_config_file = cached_file(
^^^^^^^^^^^^
File "/data/zrt/anaconda3/envs/masaenv/lib/python3.11/site-packages/transformers/utils/hub.py", line 441, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like bert-base-uncased is not the path to a directory containing a file named config.json.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.

[siyuanliii]

Thanks for your question. However, I am not able to reproduce the error on myside.. Can you first check whether you can connect to https://huggingface.co/ in your machine? Please check here for offline mode: https://github.com/open-mmlab/mmdetection/blob/main/configs/grounding_dino/README.md#note

[vitem98]

I have the same problem