CodeMan

Each of us must find a way to love the world. You have found yours.

0%

发表于 分类于

查看版本号

1
gitlab-rake gitlab:env:info

备份

1
gitlab-rake gitlab:backup:create

升级路线

官方路径
8.11.Z -> 8.12.0 -> 8.17.7 -> 9.5.10 -> 10.8.7 -> 11.11.8 -> 12.0.12 -> 12.1.17 -> 12.10.14 -> 13.0.14 -> 13.1.11 -> latest 13.12.Z -> latest 14.0.Z

自动备份目录

/etc/gitlab/config_backup

升级

当前版本：gitlab-ee-12.7.5-ee.0.el7.x86_64
目标版本：gitlab-ee-14.0.1-ee.0.el7.x86_64

1
2
3
4
5
6
7
8
9
10
11
12
13
yum install gitlab-ee-12.9.5
yum install gitlab-ee-12.10.14
yum install gitlab-ee-13.0.14
yum install gitlab-ee-13.1.11
yum install gitlab-ee-13.5.4
yum install gitlab-ee-13.6
yum install gitlab-ee-13.6.0
yum install gitlab-ee-13.8.0
yum install gitlab-ee-13.9.0
yum install gitlab-ee-13.12.0
yum install gitlab-ee-13.12.5
yum install gitlab-ee-14.0.1

每次升级过程中都会自动当前配置到配置备份目录

1
2
3
4
5
[root@ip-172-31-17-151 ~]# ls /etc/gitlab/config_backup
gitlab_config_1624679624_2021_06_26.tar  gitlab_config_1624682328_2021_06_26.tar  gitlab_config_1624683553_2021_06_26.tar  gitlab_config_1624684600_2021_06_26.tar
gitlab_config_1624680821_2021_06_26.tar  gitlab_config_1624682595_2021_06_26.tar  gitlab_config_1624683960_2021_06_26.tar  gitlab_config_1624684881_2021_06_26.tar
gitlab_config_1624681207_2021_06_26.tar  gitlab_config_1624683048_2021_06_26.tar  gitlab_config_1624684242_2021_06_26.tar

查看升级后版本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
gitlab-rake gitlab:backup:create

[root@ip-172-31-17-151 ~]# gitlab-rake gitlab:env:info

System information
System:
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.7.2p137
Gem Version:	3.1.4
Bundler Version:2.1.4
Rake Version:	13.0.3
Redis Version:	6.0.14
Git Version:	2.32.0
Sidekiq Version:5.2.9
Go Version:	unknown

GitLab information
Version:	14.0.1-ee
Revision:	38151bb98b6
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	12.4
URL:		https://code.xxx.com
HTTP Clone URL:	https://code.xxx.com/some-group/some-project.git
SSH Clone URL:	git@code.xxx.com:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers:

GitLab Shell
Version:	13.19.0
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git


注：整个升级过程不需要关闭Gitlab服务

停机

1
gitlab-ctl stop

启动

1
gitlab-ctl start

发表于 分类于

环境配置

1. 安装NODE.JS

NOJE.JS: https://nodejs.org/en/

1
2
3
4
5
6
➜  ~ node --version
v15.4.0
    
➜  ~ npm --version
7.0.15
2. 安装GIT

GIT: https://git-scm.com/downloads 

1
2
3
4

➜  ~ git --version
git version 2.28.0

3. 安装TYPESCRIPT
1
2
3
4
5
npm install --global typescript@4.2.2

➜  ~ tsc --version
Version 4.2.2

4. 安装VSCode

VS Code: https://code.visualstudio.com

创建项目

1.初始化项目
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
➜  typescript mkdir todo
➜  typescript cd todo
➜  todo npm init --yes
Wrote to /Users/yangandrew/work/typescript/todo/package.json:

{
  "name": "todo",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

2. 创建项目结构以及tsconfig.json

tsconfig.json

1
2
3
4
5
6
7
8
9
{
    "compilerOptions": {
        "target": "es2018",
        "outDir": "./dist",
        "rootDir": "./src",
        "module": "commonjs"
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
 
➜  todo mkdir dist
➜  todo mkdir src
➜  todo touch tsconfig.json
➜  todo vi tsconfig.json
➜  todo tree
.
├── dist
├── package.json
├── src
└── tsconfig.json


3. src目录下创建index.ts
1
2
3
4

console.clear();
console.log("你好，Typescript");

项目目录下编译

1
2
3
4
5
6
7
8
9
10
11
➜  todo tsc

➜  todo tree
.
├── dist
│   └── index.js
├── package.json
├── src
│   └── index.ts
└── tsconfig.json

不要去修改dist下面的js文件。它们都是由typescript编译出来的文件。

使用node运行

1
2
3
4
5
➜  todo node dist/index.js

➜  todo node dist/index.js
你好，Typescript

3. 定义数据模型

Todo需求：

  • 查年Ttem
  • 添加Ttem
  • 标记为完成
  • 查询Item

添加todoItems.ts到src目录
代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
export class TodoItem {
    public id: number;
    public task: string;
    public complete: boolean = false;
    public constructor(id: number, task: string, complete: boolean = false) {
        this.id = id;
        this.task = task;
        this.complete = complete;
    }
    public printDetails() : void {
        console.log(`${this.id}\t${this.task} ${this.complete
            ? "\t(complete)": ""}`);
    }
}

发表于 分类于

问题描述：

需要把excel中的某一列做转换大小写操作，具体代码如下：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# coding=utf-8
 
from openpyxl import load_workbook
 
 
def change_col_to_upper_or_lower(file_path='',
                                 sheet='',
                                 column='',
                                 to_upper=True,
                                 ):
    wb = load_workbook(file_path)
    ws = wb[sheet]
    max_row = ws.max_row
    min_row = ws.min_row
 
    for row_of_cell_objects in ws[column + str(min_row):column + str(max_row)]:
        for cell_obj in row_of_cell_objects:
            if to_upper:
                cell_obj.value = cell_obj.value.upper()
            else:
                cell_obj.value = cell_obj.value.lower()
    wb.save(file_path)
 
change_col_to_upper_or_lower('col_copy_tracking.xlsx','control','I')

发表于 分类于

问题描述：

Copy其中Sheet中的某一列到另一个Sheet中的某一列. Talk is cheap, here are codes.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# coding=utf-8
 
from openpyxl import load_workbook
 
 
def copy_col_to_other_sheet(file_path='',
                            from_sheet='',
                            from_column_min='',
                            from_column_max='',
                            to_sheet='',
                            to_column=''):
    wb = load_workbook(file_path)
    from_ws = wb[from_sheet]
    to_ws = wb[to_sheet]
    for row_of_cell_objects in from_ws[from_column_min:from_column_max]:
        for cell_obj in row_of_cell_objects:
            to_ws[to_column + cell_obj.coordinate[1:]].value = cell_obj.value
    wb.save(file_path)
 
 
copy_col_to_other_sheet('col_copy_tracking.xlsx', 'tracking', 'C2', 'C15', 'control', 'D')

发表于 分类于

一、环境安装见前篇Python virtualenv使用

二、包安装

考虑到要处理xlsx文件这里使用包openpyxl

1
2
3

pip  --proxy 127.0.0.1:1087 install openpyxl

三、遍历所有Cell

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# coding=utf-8

from openpyxl import load_workbook


def read_and_iterate_cell(file_path=''):
    wb = load_workbook(file_path)
    ws = wb['Latest']
    column = ws.max_column
    for row in ws.rows:
        for colIndex in range(0, column, 1):
            cell = row[colIndex]
            print(str(cell.value) + "\t"),
        print()


read_and_iterate_cell('SMS_KS_tracking_report_20191123.xlsx')

发表于 分类于

Spark是什么

Spark是通用的大数据计算框架. 如Hadoop的MapReduce、Hive引擎、Storm的流式计算引擎一样。

Spark各子项目

Spark Core：离线计算
Spark SQL: 交互式查询
Spark Streaming: 实时流式计算
Spark MLlib: 机器学习
Spark GraphX: 图计算

与Hadoop主要区别

Spark: 主要用于计算
Hadoop：主要用于大数据存储（HDFS、Hive、HBase）以及资源调度（Yarn)

Hadoop: 最耗时的部分是Shffer
Spark: 基于内存的计算

发表于 分类于

Install && Basic Usage

1
2
3
4
5
6
7
➜ ~ pip3 install virtualenv
➜ ~ mkdir project
➜ ~ cd project
➜ project virtualenv --no-site-package venv 
➜ project source venv/bin/activate 
(venv) ➜  project python --version

指定python版本

2.7:

1
2
virtualenv --no-site-package ----python=python2.7 venv

3.7:

1
2
virtualenv --no-site-package ----python=python3.7 venv

退出使用环境

(venv) ➜ project deactivate

开启代理后报错

Could not install packages due to an EnvironmentError: Missing dependencies for SOCKS support.

由于不支持sockt5，则改为http代理即可

1
2
3
unproxy
export all_proxy=https://127.0.0.1:1087

让pip使用HTTP代理

1
2
3
pip --proxy 127.0.0.1:1087 install xxx
pip --proxy 127.0.0.1:1087 install PyPDF2

Proejct Interpreter->(弹出菜单中右上角图标Add)Add…
Virtualenv Enviroment -> Existing enviroment->选择自己项目目录下venv/bin/python

tradove-v2.5-nginx:
image: 119.84.60.71:5050/release/v2.5/nginx1.15.8:release
ports:
- “${WEB_PORT}:80”
- “443:443”
depends_on:
- tradove-v2.5-php-fpm
links:
- tradove-v2.5-php-fpm:php
volumes:
- ${PROJECT_BASE_DIR}:/var/www
- ${PROJECT_BASE_DIR}/docker/nginx/certificate_dev:/etc/nginx/certificate #测试服务器可用
- ${PROJECT_BASE_DIR}/docker/nginx/conf.d:/etc/nginx/conf.d:rw
restart: always
container_name: “tradove-v2.5-nginx”
privileged: true
networks:
- tradove-net

发表于 分类于

Introduction

Two-fer or 2-fer is short for two for one. One for you and one for me.

1
2
"One for X, one for me."

When X is a name or “you”.

If the given name is “Alice”, the result should be “One for Alice, one for me.” If no name is given, the result should be “One for you, one for me.”

Solve:

Bash

version 1:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
#!/usr/bin/env bash
set -o errexit
set -o pipefail

main () {
	result="One for you, one for me."
	if [[ ! "$1" ]] ;then
		echo $result
	else
		echo "${result/you/$1}"
	fi

}

main "$@"

version 2:

1
2
3
4
5
6
7
8
9
10
11
#!/usr/bin/env bash
set -o errexit
set -o pipefail

main () {
	result="One for you, one for me."
	echo "${result/you/${1:-you}}"
}

main "$@"

相关知识点：

模板字符串：

函数参数默认值：

Php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
<?php

function twoFer($name="you")
{
    $result = 'One for {$name}, one for me.';
    return $result;
}


if (empty($argv[1])) {
    echo twoFer("you");
} else {
    echo twoFer($argv[1]);
}

Java

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
class TwoFer {
    public static void print(String msg) {
        System.out.println(msg);
    }

    public static String twoFerString(String name) {
        return String.format("One for %s, one for me.", name);
    }

    public static String twoFer(String name) {
        String defaultName = "you";

        if ((name != null) && !"".equals(name)) {
            return twoFerString(name);
        } else {
            return twoFerString(defaultName);
        }
    }

    public static void main(String[] args) {
        if ((args != null) && (args.length > 0)) {
            print(twoFer(args[0]));
        } else {
            print(twoFer(null));
        }
    }
}

Golang

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
package main

import (
	"fmt"
	"os"
)

func toFerMsg(name string) string {
	return fmt.Sprintf("One for %s, one for me.", name)
}

func twoFer(name string) string {
	defaultName := "you"
	if name == "" {
		return toFerMsg(defaultName)
	} else {
		return toFerMsg(name)
	}
}

func main() {
	if len(os.Args) > 1 {
		fmt.Println(twoFer(os.Args[1]))
	} else {
		fmt.Println(twoFer(""))
	}

}


Erlang

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

#!/usr/local/bin/escript

-module(two_fer).

-import(io, [format/2]).
-export([twofer/0, twofer/1, main/1]).

twoferMsg(N) -> io:format("One for ~s, one for me.", [N]).

twofer(N) -> twoferMsg(N).
twofer() -> twoferMsg("you").

main(Args) ->
  case Args of
    "" -> twofer();
    _ -> twofer(Args)
  end.





1
2
3
4
5
➜ chmod +x twofer.erl
➜ two_fer.erl boo
One for boo, one for me.%       
➜ two_fer.erl
One for you, one for me.%

Clojure

1
2
3
4
5
6
7
8
9
10
11
12
(ns two-fer.core
  (:gen-class))

(defn twofer [name]
  (str (format "One for %s, one for me." name)))

(defn -main
  [& args]
  (if-not (empty? args)
    (println (twofer (first args)))
    (println (twofer "you"))))

1
2
3
4
5
6
7
8
9
10
11
➜  two-fer lein uberjar                                                                                                 
Compiling two-fer.core
Created /Users/yangandrew/work/clojure/two-fer/target/uberjar/two-fer-0.1.0-SNAPSHOT.jar
Created /Users/yangandrew/work/clojure/two-fer/target/uberjar/two-fer-0.1.0-SNAPSHOT-standalone.jar

➜  two-fer java -jar /Users/yangandrew/work/clojure/two-fer/target/uberjar/two-fer-0.1.0-SNAPSHOT-standalone.jar foo boo
One for foo, one for me.

➜  two-fer java -jar /Users/yangandrew/work/clojure/two-fer/target/uberjar/two-fer-0.1.0-SNAPSHOT-standalone.jar        
One for you, one for me.

Python

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import sys

def twofer(name):
    return "One for %s, one for me." % name

def main(argv):
    if len(argv) > 1:
        print twofer(argv[1])
    else:
        print twofer("you")


if __name__ == '__main__':
    main(sys.argv)


1
2
3
4
5
(venv) ➜  twofer python twofer.py 
One for you, one for me.
(venv) ➜  twofer python twofer.py Bob
One for Bob, one for me.

发表于 分类于
基本命令
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
cat /proc/version
uname -r
uname -a
lsb_release -a
cat /etc/issue

netstat -an | grep 3306

ls -lh
du -hm --max-depth=2 | sort -nr | head -12
find . -type f -size +800M  -print0 | xargs -0 du -h | sort -nr

sudo ntsysv

awk '{print $1}' access_log |sort|uniq|wc -l
awk '{print $1}' access_log |sort|uniq -c|sort -nr |head -10
grep ^111.230.17.31 access_log | awk '{print $1,$7}' > max_acess

同步时间
1
2
3
4
5
6
7
8
9
yum -y install chrony
systemctl enable chronyd
systemctl start chronyd
timedatectl status
timedatectl set-ntp true
timedatectl
timedatectl list-timezones
timedatectl set-timezone Asia/Chongqing

Mac
1
2
3
4

lsof -i:80
sudo killall coreaudiod

发表于 分类于

Shadowsocks客户端安装

1
2
3
4
5
yum -y install epel-release
yum -y install python-pip
pip install shadowsocks
mkdir /etc/shadowsocks
vi /etc/shadowsocks/shadowsocks.json

加入配置：

1
2
3
4
5
6
7
8
9
10
11
{ 
    "server":"xxx.xxx.xxx.xxx",
    "server_port":993,
    "local_address": "127.0.0.1",
    "local_port":1080,
    "password":"xxx",
    "timeout":600,
    "method":"chacha20-ietf-poly1305",
    "fast_open": false,
    "workers": 1
}

启动脚本：

1
vi /etc/systemd/system/shadowsocks.service

内容：

1
2
3
4
5
6
7
8
9
[Unit]
Description=Shadowsocks

[Service]
TimeoutStartSec=0
ExecStart=/usr/bin/sslocal -c /etc/shadowsocks/shadowsocks.json

[Install]
WantedBy=multi-user.target

具体常用操作：

1
2
3
4
systemctl enable shadowsocks.service
systemctl start shadowsocks.service
systemctl status shadowsocks.service
systemctl stop shadowsocks.service

检查shadowsock客户端是否安装正常：

1
2
3

curl --socks5 127.0.0.1:1080 http://httpbin.org/ip

返回结果如下：

1
2
3
4
5

{  
"origin": "x.x.x.x" #你的Shadowsock服务器IP
}

安装配置privoxy

安装privoxy

1
2
3
4
5
6
yum install privoxy -y
systemctl enable privoxy
systemctl start privoxy
systemctl status privoxy
systemctl stop privoxy

配置privoxy

1
vi /etc/privoxy/config

修改配置：

1
2
listen-address 127.0.0.1:8118 # 8118 是默认端口，不用改
forward-socks5t / 127.0.0.1:1080 . #转发到本地端口，注意最
1
vi /etc/profile

加入：

1
2
3
4
5
export http_proxy=http://127.0.0.1:8118
export https_proxy=http://127.0.0.1:8118
export no_proxy=localhost,127.0.0.1

source /etc/profile

测试：

1
curl  cip.cc

ERROR method chacha20-ietf-poly1305 not supported 解决

大部分文章都说按以下命令就能解决这个问题

1
2
3

yum install libsodium -y

然而这样真没什么用，正确的解决办法是重新安装shadowsocks来解决这个问题

1
2
3
4
5
6
7
8
9
10
11
12
pip install https://github.com/shadowsocks/shadowsocks/archive/master.zip -U
Collecting https://github.com/shadowsocks/shadowsocks/archive/master.zip
  Downloading https://github.com/shadowsocks/shadowsocks/archive/master.zip (115kB)
    100% |████████████████████████████████| 122kB 871kB/s
Installing collected packages: shadowsocks
  Found existing installation: shadowsocks 2.8.2
    Uninstalling shadowsocks-2.8.2:
      Successfully uninstalled shadowsocks-2.8.2
  Running setup.py install for shadowsocks ... done
Successfully installed shadowsocks-3.0.0
You are using pip version 8.1.2, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.