InstallingMediaGoblin

From TAMI
Jump to navigation Jump to search

I'm using servus.local, Fedora 18 64 bit running on a laptop in the ""server room"".

Starting from http://docs.mediagoblin.org/siteadmin/deploying.html

  • and /usr/share/doc/postgresql-9.2.3/README.rpm-dist

install yum packages for everything except transcoding

yum install python-paste-deploy python-paste-script git-core python python-devel python-lxml python-imaging python-virtualenv -y
yum install postgresql-server postgresql python-psycopg2 -y
# For celeryd audio media type
yum install numpy -y

install of gcc, required for sqlalchemy speedups (next stage, setup.py develop)

yum install gcc -y

enable rpmfusion for ffmpeg and install gstreamer plugins for transcoding stuff

(the error below doesn't happen if you have these installed) 

su -c 'yum localinstall --nogpgcheck http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm'
yum install gstreamer-python gstreamer-plugins-good gstreamer-plugins-base gstreamer-plugins-bad gstreamer-plugins-ugly gstreamer-ffmpeg -y

setup the postgresql database

postgresql-setup initdb
systemctl enable posgresql.service
systemctl start postgresql.service

create mediagoblin user, nologin

useradd mediagoblin
vim /etc/passwd # change /bin/bash to /bin/nologin

finished setup of postgresql

verify psql works with it

su mediagoblin /bin/bash
psql
(you should get a prompt)

rest of instructions until further notice done as mediagoblin user

(i.e. don't quit the previous su, whoami should say mediagoblin)

clone and setup virtualenv

git clone git://gitorious.org/mediagoblin/mediagoblin.git
cd mediagoblin
(virtualenv --system-site-packages . || virtualenv .) && ./bin/python setup.py develop

creating our own config. enable plugins, turn on emails, use correct postgresql hostname.

cp mediagoblin.ini mediagoblin_local.ini
vim mediagoblin_local.ini

initialize database

./bin/gmg dbupdate

install nginx

yum install -y nginx
# required for "semanage permissive" (otherwise you get a failure for missing Makefile)
yum install -y selinux-policy-devel
# enable port forwarding, socket connections from httpd context are disabled by default
setsebool httpd_can_network_connect on
# This part is a hack - it's slightly better then making everything permissive (taken from man selinux_http)
semanage permissive -a httpd_t

You can get nginx to access static files by any permission fix, since there was a mediagoblin group and nginx user I added the nginx user to the mediagoblin group and so the default permissions and ownership that mediagoblin sets on the transcoded files worked (660 for mediagoblin:mediagoblin): 

vim /etc/groups
find mediagoblin
append nginx
# or equivalently: useradd -G mediagoblin nginx

create systemd unit file to start celeryd /lib/systemd/system/gmg-celeryd.service

[Unit]
Description=Mediagoblin Celeryd

#             --daemon
#             --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543' mediagoblin
 
[Service]
User=mediagoblin
Group=mediagoblin
Type=simple
WorkingDirectory=/home/mediagoblin/mediagoblin
Environment="MEDIAGOBLIN_CONFIG=/home/mediagoblin/mediagoblin/mediagoblin_local.ini" \
CELERY_CONFIG_MODULE=mediagoblin.init.celery.from_celery
ExecStart=/home/mediagoblin/mediagoblin/bin/celeryd
PIDFile=/var/run/mediagoblin/mediagoblin-celeryd.pid
[Install]
WantedBy=multi-user.target

create logging directory

mkdir /var/run/mediagoblin
chown mediagoblin:mediagoblin /var/run/mediagoblin

create systemd unit file to start paster: /lib/systemd/system/gmg.service

[Unit]
Description=Mediagoblin

[Service]
Type=forking
User=mediagoblin
Group=mediagoblin
Environment=CELERY_ALWAYS_EAGER=false
WorkingDirectory=/home/mediagoblin/mediagoblin
ExecStart=/home/mediagoblin/mediagoblin/bin/paster serve \
             /home/mediagoblin/mediagoblin/paste.ini \
             --pid-file=/var/run/mediagoblin/mediagoblin.pid \
             --log-file=/var/run/mediagoblin/mediagoblin.log \
             --daemon \
             --server-name=fcgi fcgi_host=127.0.0.1 fcgi_port=26543
ExecStop=/home/mediagoblin/mediagoblin/bin/paster serve \
            --pid-file=/var/run/mediagoblin/mediagoblin.pid \
            /home/mediagoblin/mediagoblin/paste.ini stop
PIDFile=/var/run/mediagoblin/mediagoblin.pid
[Install]
WantedBy=multi-user.target


enable and run them, check them

systemctl enable gmg-celeryd
systemctl enable gmg
systemctl start gmg-celeryd
systemctl start gmg

configure nginx to proxy to paster: use instructions in mediagoblin guide.

tmpfiles: on fedora /var is now a tmpfs

so you need to create the run directory on start. That is done either by the app or by tmpfiles.d. 

# cat /etc/tmpfiles.d/gmg.conf 
d /var/run/mediagoblin  755 mediagoblin mediagoblin

running the transcoding as mediagoblin:mediagoblin

the files are produced with rx- --- --- permissions, so nginx running as nginx:nginx cannot read them. There is probably a better solution (using setfacl maybe, I couldn't get it to work), but right now a hack on mediagoblin itself (I could also run a background process to change permissions..) to change permissions does the trick.

open http and https (TODO: document http to https redirection and certificate setup with nginx):

firewall-cmd --add-service http
firewall-cmd --add-service https

reboot and verify everything still works.

Postgres instructions

create user mediagoblin; 

\dg # show all roles, including mediagoblin, has no roles, and member of nothing

create database mediagoblin with owner = mediagoblin; 

postgres=# \dg
                             List of roles
 Role name  |                   Attributes                   | Member of

+------------------------------------------------+----------- 

mediagoblin |                                                | {}

postgres=# \l
                                   List of databases
   Name     |    Owner    | Encoding |   Collate   |    Ctype    |   Access privileges

+-------------+----------+-------------+-------------+----------------------- 

mediagoblin | mediagoblin | UTF8     | en_US.UTF-8 | en_US.UTF-8 |

After "./bin/gmg dbupdate" you should have the following tables: 

mediagoblin=> \dt

                   List of relations
Schema |           Name            | Type  |    Owner

+---------------------------+-------+------------- 

public | core__attachment_files    | table | mediagoblin
public | core__collection_items    | table | mediagoblin
public | core__collections         | table | mediagoblin
public | core__file_keynames       | table | mediagoblin
public | core__media_comments      | table | mediagoblin
public | core__media_entries       | table | mediagoblin
public | core__media_tags          | table | mediagoblin
public | core__mediafiles          | table | mediagoblin
public | core__migrations          | table | mediagoblin
public | core__processing_metadata | table | mediagoblin
public | core__tags                | table | mediagoblin
public | core__users               | table | mediagoblin
public | image__mediadata          | table | mediagoblin
public | video__mediadata          | table | mediagoblin
(14 rows)

Bugs

Transcording problem, this was before adding rpmfusion and installing gstreamer-ffmpeg, gstreamer-plugins-bad, gstreamer-plugins-ugly: 

2013-03-16 15:55:33,412 DEBUG   [mediagoblin.processing.task] Processing <MediaEntry 3: don't know yet>
2013-03-16 15:55:33,415 INFO    [mediagoblin.media_types.video.transcoders] Initializing VideoTranscoder...
2013-03-16 15:55:33,416 INFO    [mediagoblin.media_types.video.transcoders] Discovering /home/mediagoblin/mediagoblin/user_dev/media/queue/media_entries/bce350d1-9d89-4ee3-b1c0-15193b90fb88/13030052.mp4
Traceback (most recent call last):
  File "/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/transcoders.py", line 726, in __on_discovered
    raise Exception('Could not discover {0}'.format(self.source_path))
Exception: Could not discover /home/mediagoblin/mediagoblin/user_dev/media/queue/media_entries/bce350d1-9d89-4ee3-b1c0-15193b90fb88/13030052.mp4
2013-03-16 15:55:33,445 INFO    [mediagoblin.media_types.video.transcoders] Terminating MainLoop
2013-03-16 15:55:33,446 ERROR   [mediagoblin.processing.task] An unhandled exception was raised while processing <MediaEntry 3: don't know yet>
2013-03-16 15:55:33,447 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",)
2013-03-16 15:55:33,502 WARNING [mediagoblin.processing] No idea what happened here, but it failed: TypeError("argument of type 'NoneType' is not iterable",)
Error - <type 'exceptions.TypeError'>: argument of type 'NoneType' is not iterable
URL: http://servus.local:6543/submit/
File '/usr/lib/python2.7/site-packages/paste/exceptions/errormiddleware.py', line 144 in __call__
  app_iter = self.application(environ, sr_checker)
File '/usr/lib/python2.7/site-packages/paste/urlmap.py', line 203 in __call__
  return app(environ, start_response)
File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/Beaker-1.5.4-py2.7.egg/beaker/middleware.py', line 152 in __call__
  return self.wrap_app(environ, session_start_response)
File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 236 in __call__
  return self.call_backend(environ, start_response)
File '/home/mediagoblin/mediagoblin/mediagoblin/app.py', line 216 in call_backend
  response = controller(request)
File '/home/mediagoblin/mediagoblin/mediagoblin/decorators.py', line 48 in new_controller_func
  return controller(request, *args, **kwargs)
File '/home/mediagoblin/mediagoblin/mediagoblin/submit/views.py', line 96 in submit_start
  run_process_media(entry, feed_url)
File '/home/mediagoblin/mediagoblin/mediagoblin/submit/lib.py', line 68 in run_process_media
  task_id=entry.queued_task_id)
File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 445 in apply_async
  return self.apply(args, kwargs, task_id=task_id, **options)
File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/app/task/__init__.py', line 601 in apply
  request=request, propagate=throw)
File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 248 in eager_trace_task
  uuid, args, kwargs, request)
File '/home/mediagoblin/mediagoblin/lib/python2.7/site-packages/celery-2.5.3-py2.7.egg/celery/execute/trace.py', line 181 in trace_task
  R = retval = fun(*args, **kwargs)
File '/home/mediagoblin/mediagoblin/mediagoblin/processing/task.py', line 92 in run
  entry.media_manager['processor'](proc_state)
File '/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/processing.py', line 91 in process_video
  store_metadata(entry, metadata)
File '/home/mediagoblin/mediagoblin/mediagoblin/media_types/video/processing.py', line 170 in store_metadata
  if key in metadata])
TypeError: argument of type 'NoneType' is not iterable


CGI Variables
-------------
  CONTENT_LENGTH: '7497175'
  CONTENT_TYPE: 'multipart/form-data; boundary=---------------------------185490103311094336141507406595'
  CSRF_TOKEN: u'18270768167913598379'
  HTTP_ACCEPT: 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8'
  HTTP_ACCEPT_ENCODING: 'gzip, deflate'
  HTTP_ACCEPT_LANGUAGE: 'en-US,en;q=0.5'
  HTTP_CONNECTION: 'keep-alive'
  HTTP_COOKIE: 'mediagoblin_csrftoken=18270768167913598379; mediagoblin_csrftoken=18270768167913598379; mediagoblin=57de25a836140d810f8c9d5ed58d2bc9'
  HTTP_DNT: '1'
  HTTP_HOST: 'servus.local:6543'
  HTTP_REFERER: 'http://servus.local:6543/submit/'
  HTTP_USER_AGENT: 'Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20100101 Firefox/19.0'
  PATH_INFO: '/submit/'
  REMOTE_ADDR: '10.81.2.190'
  REQUEST_METHOD: 'POST'
  SERVER_NAME: '0.0.0.0'
  SERVER_PORT: '6543'
  SERVER_PROTOCOL: 'HTTP/1.1'


WSGI Variables
--------------
  application: {(None, ): <beaker.middleware.SessionMiddleware object at 0x41848d0>, (None, '/mgoblin_static'): <StaticURLParser '/home/mediagoblin/mediagoblin/mediagoblin/static'>, (None, '/theme_static'): <StaticURLParser '/home/mediagoblin/mediagoblin/user_dev/theme_static'>, (None, '/mgoblin_media'): <StaticURLParser '/home/mediagoblin/mediagoblin/user_dev/media/public'>}
  beaker.get_session: <bound method SessionMiddleware._get_session of <beaker.middleware.SessionMiddleware object at 0x41848d0>>
  beaker.session: {'user_id': u'1', '_accessed_time': 1363442133.226504, 'messages': [], '_creation_time': 1363439039.214583}
  paste.httpserver.thread_pool: <paste.httpserver.ThreadPool object at 0x28bf650>
  paste.throw_errors: True
  werkzeug.request: <Request 'http://servus.local:6543/submit/' [POST]>
  wsgi process: 'Multithreaded'
------------------------------------------------------------
Retrieved from "https://wiki.telavivmakers.org/index.php?title=InstallingMediaGoblin&oldid=3298"