【1.6】JupyterHub——让Jupyter支持多用户
支持多用户的jupyter
一、安装
1.下载并安装Anaconda3
wget -c https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh
bash Anaconda3-5.2.0-Linux-x86_64.sh
安装提示:
Do you wish the installer to prepend the Anaconda3 install location
to PATH in your /root/.bashrc ? [yes|no]
[no] >>> /data/user/sam/.bashrc
修改环境变量
vim /etc/profile
# Anaconda 3
#export PATH=/data/software/anaconda3/bin:$PATH #因为现在默认的是anaconda2里面的python2
alias activate3="source /data/software/anaconda3/bin/activate root"
alias deactivate3="source /data/software/anaconda3/bin/deactivate"
source /etc/profile
2.下载安装 jupyterhub
source /data/software/anaconda3/bin/activate root #启动conda环境,如无提示,后续的操作,都需要在该环境下完成
conda install -c conda-forge jupyterhub
3、创建配置文件
# source /data/software/anaconda3/bin/activate root 在conda环境下运行
jupyterhub --generate-config
提示:此时的工作路径是/data/software
Writing default config to: jupyterhub_config.py
4、修改配置文件
vim jupyterhub_config.py
取消注释,根据注释提示,填写相关信息
c.JupyterHub.ip = 'IP地址'
c.Spawner.ip = '127.0.0.1'
c.JupyterHub.port = 6364
c.PAMAuthenticator.encoding = 'utf8'
c.LocalAuthenticator.create_system_users = True
c.Authenticator.whitelist = {'user1', 'user1', 'user3'}
c.Authenticator.admin_users = {'user1'}
c.LocalAuthenticator.group_whitelist = {'group1'}
c.JupyterHub.statsd_prefix = 'jupyterhub'
组白名单好像会让用户白名单失效。所以需要确保用户在组白名单里面,需要检查一下/etc/group
5、启动JupyterHub
# source /data/software/anaconda3/bin/activate root 在conda环境下运行
nohup jupyterhub --no-ssl > jupyterhub.log &
(在配置文件目录下执行此命令)
8、测试访问
用IP+端口测试访问
9、用户管理
用户白名单的用户会自动添加,但无密码,需要修改密码才能登录;
adduser等
二、讨论
2.1.同时支持Python2和python3
Notebook的右上角点new 只看到 python 3 kernel,需要同时支持Python2和python3
查看目前的conda环境中的kernels
[root@pydev pydev]# jupyter-kernelspec list
Available kernels:
python3 /data/software/anaconda3/share/jupyter/kernels/python3
查看是否安装了python2
python2 -m pip --version
然后,在该conda环境下通过Python2关联ipykernel
python2 -m pip install --upgrade ipykernel
python2 -m pip install ipykernel
python2 -m ipykernel install --user
source activate anaconda2 ##这个是启动py2的conda环境
ipython kernel install
查看该环境下的kernels
# source /data/software/anaconda3/bin/activate root 在conda环境下运行
jupyter-kernelspec list
Available kernels:
python2 /root/.local/share/jupyter/kernels/python2
python3 /data/software/anaconda3/share/jupyter/kernels/python3
重启,打开jupyterhub,就能看到py2和py3的切换了
2.2 同步Linux的环境变量
jupyterhub启动以后,linux的path中新增的部分,发现没法在os.environ[‘PATH’]找到,最后不得已重启jupyterhub。
ps -ef |grep jupy
kill -9 process_id11111
cd /data/sotware # 在配置文件目录下执行
source /data/software/anaconda3/bin/activate root
nohup jupyterhub --no-ssl > jupyterhub.log &
同时,修改配置文件,限制单个用户的环境变量不要出现
vim /data/software/jupyterhub_config.py
c.Spawner.env_keep = ['PATH', 'PYTHONPATH', 'CONDA_ROOT', 'CONDA_DEFAULT_ENV', 'VIRTUAL_ENV', 'LANG', 'LC_ALL']
##这里是想要单个用户中出现的环境变量
c.Spawner.environment = {}
三、报错
报错1 : configurable-http-proxy没有访问权限
解决办法:
升级: npm install -g configurable-http-proxy
(base) [sam@c01 software]# jupyterhub --no-ssl
[I 2018-12-10 16:35:39.200 JupyterHub app:1667] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.2
[I 2018-12-10 16:35:39.200 JupyterHub app:1667] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.2
[I 2018-12-10 16:35:39.210 JupyterHub app:1014] Loading cookie_secret from /mnt/nfs/data/software/jupyterhub_cookie_secret
[I 2018-12-10 16:35:39.355 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2018-12-10 16:35:39.361 JupyterHub auth:490] Ignoring username whitelist because group whitelist supplied!
[I 2018-12-10 16:35:39.583 JupyterHub app:1849] Hub API listening on http://127.0.0.1:8081/hub/
[W 2018-12-10 16:35:39.588 JupyterHub proxy:560] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
[I 2018-12-10 16:35:39.588 JupyterHub proxy:562] Starting proxy @ http://*******25:6364/
[C 2018-12-10 16:35:39.601 JupyterHub app:1861] Failed to start proxy
Traceback (most recent call last):
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/app.py", line 1859, in start
await self.proxy.start()
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/proxy.py", line 566, in start
self.proxy_process = Popen(cmd, env=env, start_new_session=True, shell=shell)
File "/data/software/anaconda3/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/data/software/anaconda3/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: 'configurable-http-proxy'
(base) [root@c01 software]# npm install -g configurable-http-proxy
/mnt/nfs/data/software/anaconda3/bin/configurable-http-proxy -> /mnt/nfs/data/software/anaconda3/lib/node_modules/configurable-http-proxy/bin/configurable-http-proxy
+ configurable-http-proxy@4.0.1
added 33 packages and updated 7 packages in 11.264s
╭─────────────────────────────────────╮
│ │
│ Update available 5.6.0 → 6.4.1 │
│ Run npm i npm to update │
│ │
报错2: (未解决)
(base) [sam@c01 software]$ jupyterhub --no-ssl
[I 2019-03-04 11:33:20.137 JupyterHub app:1667] Using Authenticator: jupyterhub.auth.PAMAuthenticator-0.9.2
[I 2019-03-04 11:33:20.137 JupyterHub app:1667] Using Spawner: jupyterhub.spawner.LocalProcessSpawner-0.9.2
[I 2019-03-04 11:33:20.400 JupyterHub app:1014] Loading cookie_secret from /mnt/nfs/data/software/jupyterhub_cookie_secret
[I 2019-03-04 11:33:22.226 JupyterHub proxy:431] Generating new CONFIGPROXY_AUTH_TOKEN
[W 2019-03-04 11:33:22.286 JupyterHub auth:490] Ignoring username whitelist because group whitelist supplied!
[I 2019-03-04 11:33:23.390 JupyterHub app:1849] Hub API listening on http://127.0.0.1:8081/hub/
[W 2019-03-04 11:33:23.403 JupyterHub proxy:472] Found proxy pid file: /mnt/nfs/data/software/jupyterhub-proxy.pid
[W 2019-03-04 11:33:23.470 JupyterHub proxy:489] Proxy still running at pid=56607
[W 2019-03-04 11:33:24.473 JupyterHub proxy:504] Stopped proxy at pid=56607
[W 2019-03-04 11:33:24.502 JupyterHub proxy:560] Running JupyterHub without SSL. I hope there is SSL termination happening somewhere else...
[I 2019-03-04 11:33:24.502 JupyterHub proxy:562] Starting proxy @ http://19***:6364/
[C 2019-03-04 11:33:35.330 JupyterHub app:1861] Failed to start proxy
Traceback (most recent call last):
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/app.py", line 1859, in start
await self.proxy.start()
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/proxy.py", line 596, in start
await server.wait_up(1)
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/utils.py", line 165, in wait_for_server
timeout=timeout
File "/data/software/anaconda3/lib/python3.6/site-packages/jupyterhub/utils.py", line 155, in exponential_backoff
raise TimeoutError(fail_message)
TimeoutError: Server at 192.168.50.125:6364 didn't respond in 1 seconds
(base) [sam@c01 software]$ 11:33:37.915 [ConfigProxy] info: Proxying http://1*****:6364 to (no default)
11:33:37.925 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
解决路径:
修改了start_timeout
vim /data/software/jupyterhub_config.py
c.Spawner.start_timeout = 60
接着报错;
14:20:47.424 [ConfigProxy] info: Proxy API at http://127.0.0.1:8001/api/routes
14:20:48.738 [ConfigProxy] error: 404 GET /
14:20:48.760 [ConfigProxy] error: Failed to get custom error page: Error: connect ECONNREFUSED 127.0.0.1:8081
14:20:48.798 [ConfigProxy] error: 404 GET /favicon.ico
14:20:48.801 [ConfigProxy] error: Failed to get custom error page: Error: connect ECONNREFUSED 127.0.0.1:8081
关闭已开启的configurable-http-proxy服务
(base) [sam@c01 software]$ ps -ef |grep configurable-http-proxy
sam 30428 1 0 14:20 ? 00:00:01 node /data/software/anaconda3/bin/configurable-http-proxy --ip 1****5 --port 6364 --api-ip 127.0.0.1 --api-port 8001 --error-target http://127.0.0.1:8081/hub/error
[root@c01 ~]# netstat -lnp |grep 8001
tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 25354/node
kill -9 30428
可是还是一直报错,无力解决了,最后切换了一台服务器,就神奇般的好了,真是amazing !
报错3
[D 2017-08-29 17:27:43.001 JupyterHub proxy:490] Proxy started and appears to be up
[D 2017-08-29 17:27:43.003 JupyterHub proxy:551] Proxy: Fetching GET http://127.0.0.1:8001/api/routes
17:27:43.041 - warn: [ConfigProxy] 403 GET /api/routes
[E 2017-08-29 17:27:43.040 JupyterHub app:1626]
Traceback (most recent call last):
File "/home/fmilano/jupyterhub/jupyterhub/app.py", line 1624, in launch_instance_async
yield self.start()
File "/home/fmilano/jupyterhub/jupyterhub/app.py", line 1572, in start
yield self.proxy.check_routes(self.users, self._service_map)
File "/home/fmilano/jupyterhub/jupyterhub/proxy.py", line 293, in check_routes
routes = yield self.get_all_routes()
File "/home/fmilano/jupyterhub/jupyterhub/proxy.py", line 588, in get_all_routes
resp = yield self.api_request('', client=client)
tornado.httpclient.HTTPError: HTTP 403: Forbidden
解决办法:
ps aux | grep configurable-http-proxy
然后,Kill掉以后,重启服务
报错4
登陆,提示用户名不存在或者密码不对。
-
检查密码是否正确,重置密码以后发现仍然不能登陆
-
检查/data/software/jupyterhub.log 文件,提示用户不在白名单中
-
/data/software/jupyterhub_config.py中的whitelist中加入该用户,发现依旧提示用户不在白名单中。
-
确保用户名所在的用户组为group_whitelist中的用户,确保/etc/group中的用户组中,包含该用户。问题解决。
-
上述如果都没问题,建议用root权限来jupyterhub
[#root] source /data/software/anaconda3/bin/activate root 在conda环境下运行 [#root] nohup jupyterhub –no-ssl > jupyterhub.log &
这个问题主要原因是,用户不在/etc/group中对应的用户组里面,而jupyterhub中的白名单组会取代白名单用户,只有白名单组生效。。
报错5
不提示错误,但一直不能访问。
有可能是防火墙打开,把端口给禁掉了。
报错6
500 : Internal Server Error
解决办法:
输入如下命令,安装升级nbconvert
pip install --upgrade --user nbconvert
成功安装完nbconvert后再次启动jupyter notebook,在浏览器就可以正常打开.ipynb文件。
发现还是有报错:
PermissionError: [Errno 13] Permission denied: '/data/software/miniconda3/share/jupyter/nbconvert/templates/html/conf.json'
发现是这个文件夹权限的问题:
[root@c01 ~]# chmod 777 /data/software/miniconda3/share/jupyter/nbconvert/templates/html
大公告及
参考资料
个人公众号,比较懒,很少更新,可以在上面提问题,如果回复不及时,可发邮件给我: tiehan@sina.cn