Skip to content

There are little tricks about using Ubuntu that can greatly enhance your efficiency and productivity. Learning by doing!

Notifications You must be signed in to change notification settings

iamstarlee/Ubuntu-Tricks

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

66. 添加.gitignore文件可以忽略掉不想上传的更改

target          //忽略这个target目录
angular.json    //忽略这个angular.json文件
log/*           //忽略log下的所有文件
css/*.css       //忽略css目录下的.css文件

65. WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f891079f160>: Failed to establish a new connection: [Errno 111] 拒绝连接'))': /simple/ninja/

export http_proxy=''
export https_proxy=''

unset all_proxy
unset ALL_PROXY

64. 多卡推理

问:ValueError: Cannot assign non-leaf Tensor to parameter 'weight'. Model parameters must be created explicitly. To express 'weight' as a function of another Tensor, compute the value in the forward() method.
答:指定一张显卡:CUDA_VISIBLE_DEVICES=0 python predict.py

63. !pip install -q xxx

-q: stands for 'quiet mode', which suppresses most of the output during installation.
!: This is used to run shell commands from within a Jupyter notebook.

62. nohup

tail -1000 nohup.out  (查看最后1000行日志文本) 
tail -f nohup.out   (监控日志打印)

查看nohup.out最后1000行数据

61. 杀死stopped process

kill -9 `jobs -ps`

jobs -ps lists the process IDs (-p) of the stopped (-s) jobs.

60. 导出和安装anaconda环境

使用pip

pip freeze > requirements.txt
pip install -r requirements.txt

使用conda

conda env export > environment.yaml
conda env create -f environment.yaml

59 python print带颜色输出

print("\n\033[1;33;44m温馨提示,head部分没有载入是正常现象,Backbone部分没有载入是错误的。\033[0m")
  • \033 This is the escape character, which begins the ANSI color code sequence.
  • 1 : This specifies bold text.
  • 33 : This sets the text color to yellow.
  • 44 : This sets the background color to blue.
  • m : This marks the end of the color code and applies the style.
  • \033[0m: This resets the color formatting back to the default after the message, ensuring that subsequent text is not affected by this style.

58. torch.where(condition, input, other)

根据判断条件选择替换元素
Return a tensor of elements selected from either input or other, depending on condition.
如果x的元素大于0,就替换为1.0,否则就替换为0.0

torch.where(x > 0, 1.0, 0.0)

57. 环境变量的应用

在运行 Python 脚本的命令前加上环境变量设置,这样环境变量会应用于该次运行!

CUDA_VISIBLE_DEVICE=0 python main.py

56. python格式化输出f-string

给所有keys值添加backbone前缀!

renamed_dict = {f'backbone.{k}': v 
                for k, v in pretrained_dict.items()}

55. Linux查看进程

htop 是一个 Linux 下的交互式的进程浏览器,可以用来替换Linux下的top命令。

apt-get install htop
htop

54. Ubuntu统计文件夹内的文件数量

  • 统计当前目录下文件的个数(不包括目录)
ls -l | grep "^-" | wc -l
  • 统计当前目录下文件的个数(包括目录)
ls -lR | grep "^-" | wc -l
  • 查看某目录下文件夹(目录)的个数(包括子目录)
ls -lR | grep "^d" | wc -l

53. 下划线在python中的用途

Python 中下划线的 5 种含义

52. import os重命名

os.rename(old_file_path, new_file_path)

51. Half and Float dtype

Half means dtype = torch.float16 while, Float means dtype = torch.float32

50. What does 'register_buffer' do ?

If you have parameters in your model, which should be saved and restored in the state_dict, but not trained by the optimizer, you should register them as buffers.

49. torch.to(device)

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

48. Windows 查看文件夹中的所有文件信息

dir /b 查看当前路径下的所有文件和文件夹
dir /b /s 还可以查看子目录下的文件

47 获取sequential的模组名字

module = Sequential()
names = module.__class__.__nmae__

46. git撤回撤回

查看log

git reflog

回到特定commit

git reset --hard commitid

or

git reset --hard HEAD@{2} # {}中表示的是结点的序号

45. git撤回

撤回commit

git reset --soft HEAD^

撤回add

git reset --hard HEAD^

44. zip and unzip in ubuntu

见md文件

43. 关闭anaconda自动激活base环境导致环境有俩括号

conda config --set auto_activate_base false

image image

42. Ubuntu环境配置

  1. wget安装anaconda3
wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-5.3.1-Linux-x86_64.sh
bash Anaconda3-5.3.1-Linux-x86_64.sh
  1. pip安装torch2.3.0+cu121和torchvision0.18.0+cu121
pip install torch==2.3.0+cu121 torchvision==0.18.0+cu121 torchaudio==2.3.0+cu121 --index-url https://download.pytorch.org/whl/cu121
pip install torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1+cu118 -f https://download.pytorch.org/whl/torch_stable.html

41. 查看内存和磁盘空间

free -m
df -h

40. 同时unzip多个zip文件

for file in train*.zip; do unzip "$file" -d /path/to/destination; done

39. 打印模型sequential的中间结果

def printInfo(self):
    x = torch.rand([3,512,1,1])
    for name, module in self.fc.named_children():
        x = module(x)
        if isinstance(module,nn.Upsample):
            print("Upsample({}) : {}".format(name,x.shape))
        elif isinstance(module,nn.Conv2d):
            print("Conv2d({}) : {}".format(name,x.shape))

38. Ubuntu 查看源

cat /etc/apt/sources.list

cat(英文全拼:concatenate)命令用于连接文件并打印到标准输出设备上,它的主要作用是用于查看和连接文件。

37. Windows tricks 设置开机自启

  1. Win + r
  2. shell:startup
  3. 把快捷方式拖进来

36 创建软连接和删除软连接!

ln -sf ../COCO/coco-pose ./dataset/coco-pose (后面dataset里面没有coco-pose,软连接会从前面链接一个coco-pose过来)
unlink ./dataset/coco-pose #最好不用rm删

36.5 python3软链接到python

ln -s /usr/bin/python3 /usr/bin/python

35. 激活灵汐

source lyngor1.17/bin/activate

34. Windows与Linux系统在同一个局域网下互传文件

# 复制 Windows 文件到 Linux
scp D:\data\1.txt [email protected]:/root/data
# 复制 Windows 目录到 Linux(记得加 -r)
scp -r D:\data [email protected]:/root/data
 
# 复制 Linux 文件到 Windows
scp [email protected]:/root/data/1.txt D:\data
# 复制 Linux 目录到 Windows(记得加 -r)
scp -r [email protected]:/root/data D:\data

33. Look for version of Jetson Xavier NX

sudo apt-cache show nvidia-jetpack

32. Openpose

Webcam

31. 多线程编译

make -j`nproc`

30. 用清华源下载特定包或者一定范围的包

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple  " pyqtwebengine&lt<5.13"
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple  pathlib

29. Ubuntu终端的复制粘贴

Ctrl + Shift + C
Ctrl + Shift + V

28. ubuntu安装卸载软件

sudo dpkg -i xxx.deb
sudo dpkg -r xxx

27. 安装缺少依赖

sudo apt-get -f install

26. numpy setting

import numpy as np

# 保证所有数据能够显示,而不是用省略号表示,np.inf表示一个足够大的数
np.set_printoptions(threshold = np.inf) 

# 若想不以科学计数显示:
np.set_printoptions(suppress = True)

25. off-the-shelf

example

关于torch.nn.Embedding通透的解释

22. Use nproc to know the number of cpu cores, or use make -j${nproc}

21. Ubuntu下的文件解压

tar -xvJf xxx.tar.xz
tar -zxvf xxx.tgz

19. Ubuntu配置软链接

sudo ln -s /cuda/cuda118 /usr/local/cuda

18. Ubuntu文件或文件夹有锁

sudo chmod -R 777 文件夹或文件的路径

17. 将远端仓库同步到本地:git fetch + git merge == git pull

14. 国内NVIDIA镜像地址

  1. https://gitlab.com/nvidia/container-images/cuda/blob/master/doc/supported-tags.md
  2. https://docker.aityp.com/r/docker.io/nvidia/cuda

13. How can I find out my user name?

whoami

11. Good tutorial to install opencv in Ubuntu

Ubuntu 20.04搭建OpenCV 4.5.0 & C++环境

10. Cool profiles of Github

  1. 如何拥有一款有特色的 Github Profile?
  2. Github 首页美化教程 —— 美,是第一生产力
  3. GitHub美化主页设计(保姆教程 && Markdown表情)

9. What does 'register_buffer' do in pytorch ?

If you have parameters in your model, which should be saved and restored in the state_dict, but not trained by the optimizer, you should register them as buffers. Buffers won’t be returned in model.parameters(), so that the optimizer won’t have a change to update them.

8. pip下载指定镜像地址,或者直接设置清华源

pip install xxx -i https://pypi.tuna.tsinghua.edu.cn/simple
pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple

7. python导入其他文件夹中的包

最好加在文件中的第一行

import sys
sys.path.append('/path/to/your/module')

6.5

How to quick search issues involving yourselves. 在GitHub搜索中放入is:issue involves:my-username

6. ffmpeg将多个MP4视频合并为一个

  1. 新建一个vdeos.txt文件,文件内容是需要合并的视频名称
  2. 执行ffmpeg指令
file 'test.mp4'
file 'test2.mp4'

ffmpeg -f concat -i videos.txt -c copy concat.mp4

5. ffmpeg将多张图片合并为一个MP4

ffmpeg -f image2 -i %d.jpeg output.mp4

4. ffmpeg将视频调整成目标帧率

ffmpeg -i input1.mp4 -r 30 -vf "fps=30" temp1.mp4 其中,-r 30 指定目标帧率为 30 帧每秒,并使用 -vf "fps=30" 过滤器确保输出视频的帧率为 30 帧每秒。

3. Sublime 配置 C++11

Sublime config C++

2. ViT:使用HuggingFace和Pytorch对Vision Transformer进行微调实战

微调ViT的教程

1. 关于Ubuntu的使用细节

查看幻灯片的播放速度

gsettings get org.yorba.shotwell.preferences.slideshow delay
gsettings get org.yorba.shotwell.preferences.slideshow show-title
gsettings get org.yorba.shotwell.preferences.slideshow transition-delay

修改幻灯片的播放速度

gsettings set org.yorba.shotwell.preferences.slideshow delay 1
gsettings set org.yorba.shotwell.preferences.slideshow show-title true
gsettings set org.yorba.shotwell.preferences.slideshow transition-delay 0.1

0. Conda激活失效

source ~/anaconda3/etc/profile.d/conda.sh

About

There are little tricks about using Ubuntu that can greatly enhance your efficiency and productivity. Learning by doing!

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published