Jamroom Logo Jamroom 5 Core
is now Open Source!
User Support Forum Archive (Read Only)
Jamroom Help:
jrImport module beta testing
Paul
Jamroom Team


Joined: 20 Aug 2003
Posts: 5341
Location: Nottingham, UK

Posted: 06/14/13 06:59 
I'm looking for 3-4 beta testers that I can work directly with - You should have a smaller JR4 site (say under 10gigs of media) and an equal amount of free disk space on your fresh JR5 install to import to.
Below is the 'ReadMe' file of the jrImport module so that you have some idea of what is involved.
Please PM me if you want to help.
Thanks
Paul


Description

Provides a Jamroom5 tool to import items (songs, videos, blogs etc.) from a Jamroom4 site into a Jamroom5 site.

Installation

Copy this module and all its files and folders to the Jamroom5 modules directory.

Jamroom4 database info and files are sourced via the custom module jrExport supplied with this module in the Jamroom4_jrExport_Module folder. Install this module on the Jamroom4 system in the normal way. If you have cluster servers, be sure to synchronise them so that the jrExport module is copied to all clusters.

Communication with the Jamroom4 module is verified via a key code. In jrExport settings, enter an alphanumeric key of your choice.

To configure this jrImport module, click on its Global Config tab. The help buttons against each field provides full information on what is required.

Note the quota selection field(s) that define how Jamroom4 quotas are mapped to Jamroom5 quotas. If you want your Jamroom4 artists and members to be imported to equivalent Jamroom5 quotas, you need to create them prior to import.

When saving this form, jrImport attempts a test communication with the Jamroom4 jrExport module and reports the result.

Usage

Click on the module's Tools tab, then the Import button.

There is an extensive list of checkbox options to control the importing of all Jamroom4 items. The help button against each checkbox gives more information.

All checkboxes are enabled by default. Leave them as such and run the import process. Hopefully all will go well and you shouldn't need to uncheck any of these options before retrying (see below).

Jamroom4 database tables for each item are read, looped through, tested for validity and the equivalent Jamroom5 items are created. Any associated media files are copied to the correct Jamroom5 profile folder.

Import progress is reported in a modal window. The same progress information is written to the /data/logs/jrImport_log file.

Troubleshooting

Transferring data and files (particularly large media files) over the internet is hazardous!! Its not uncommon to encounter timeouts and other errors that can cause this import process to hang. If there is no import progress for more than a few minutes, refresh the page and click the Reset Progress Table help button for what to do if/when this happens.

Caveats

It is highly recommended that Jamroom4 sites are only imported to new, freshly installed Jamroom5 sites so as to avoid profile, user and item conflicts

As all Jamroom4 media files are copied to the Jamroom5 system, please ensure that you have enough disk space for them all.

The whole of the import process takes time, particularly when copying media files. One hour per 10 Gigabytes of data is not uncommon. Please be patient.

A good knowledge of both Jamroom4 and Jamroom5 systems is essential for understanding and using this module.

MOST IMPORTANT: Please be aware that Jamroom4 and Jamroom5 systems are completely different and not 100% compatible. Not all Jamroom4 item data may be imported or be immediately visible in the Jamroom5 system. jrImport will import Jamroom4 items to compatible Jamroom5 modules, not necessarily equivalent, eg. Channels and Stations are converted to Playlists. Not all Jamroom4 media files are used by Jamroom5, eg. Playlists have no associated image, so Jamroom4 Channel and Station images are lost. Also, Jamroom4 user images are not imported, but copied from the Jamroom5 Profile image.

Jamroom4 song and video items for sale as vault items will have their price set in Jamroom5, but be aware that the download file upon sale is the original media file, not the optionally uploaded Jamroom4 vault file.

Store, photo and event (tickets) Jamroom4 vault items are not imported. Other Jamroom4 vault items are imported to the jrFile module and sold from there (please ensure that jrFile is configured to accept the required file extensions).

No Jamroom4 sales history is imported.

No Jamroom4 activity or error logs are imported.

Import Service

If all of this seems a bit daunting, Jamroom Network offers an import service. Please visit http://jamroomnetwork.com/xxxx for more information.


_________________
Paul Asher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
Dazed



Joined: 22 Mar 2006
Posts: 4147

Posted: 06/15/13 16:56 
Hey Paul - Quick question for you. During the conversion process from JR4 to JR5, you mention having an equal amount of free disk space. Is that for the beta testing only or will that be true for the actual conversion process? That will impact my server search in regards to hard drive space.

Thanks

Back to top
Paul
Jamroom Team


Joined: 20 Aug 2003
Posts: 5341
Location: Nottingham, UK

Posted: 06/16/13 02:26 

Dazed:
Hey Paul - Quick question for you. During the conversion process from JR4 to JR5, you mention having an equal amount of free disk space. Is that for the beta testing only or will that be true for the actual conversion process? That will impact my server search in regards to hard drive space.

Thanks

It will be so for the conversion process.
As you know, JR4 and 5 are totally different. JR4 files are copied to the JR5 site and the JR5 database tables are created in the process.
I could have deleted the JR4 files during the process, to conserve disk space, but that would mean no going back or retrying should the process go wrong, for whatever reason, or you didn't initially set JR5 up to you liking and you needed to do it again.
Once you are happy that everything has been imported to the new JR5 site, its ok to delete the JR4 site.
So, if your JR4 site is 10GBytes, say, make sure you have 20GBytes of hosting space, install JR5 in a sub-domain, import to it, be happy with the result, delete JR4, move the subdomain and JR5 up into the empty JR4 space.
Alternatively, you can build the new JR5 on a new host, import to it, then either keep it there or copy it back to the original host.
Hope that makes sense.


_________________
Paul Asher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
perfectgamestudios



Joined: 27 Apr 2005
Posts: 698

Posted: 06/16/13 22:56 
So far seems to work well for me and great Job only issue I can see is maybe we could add option to not import empty profiles without image or music etc

and maybe utf8 conversion on the text as I see some profiles we strange symbols caused by the text is not utf8

Stephen

Back to top
Paul
Jamroom Team


Joined: 20 Aug 2003
Posts: 5341
Location: Nottingham, UK

Posted: 06/17/13 01:19 
Thanks Stephen
I'll look into adding an option not to import profiles with no items (or image). It will be a good opportunity for admins to prune their sites during the upgrade process.
I'll also checkout the utf8 conversion.
Can you email me your JR4 site url and the key you set so that I can try importing it to one of my test sites to see how it goes from my end?
Cheers
Pa


_________________
Paul Asher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
djmerlyn
Jamroom Ustad


Joined: 18 Dec 2003
Posts: 13497
Location: Behind You

Posted: 06/17/13 03:19 

Dazed:
Hey Paul - Quick question for you. During the conversion process from JR4 to JR5, you mention having an equal amount of free disk space. Is that for the beta testing only or will that be true for the actual conversion process? That will impact my server search in regards to hard drive space.

Thanks


I'm pretty sure what he's saying is that any site looking to upgrade using this will require 2x the disk space that is currently being used for the move- then you'd be able to free it up by removing the old site.

It would be neat if it could do a move instead of copy. Moving a file takes a fraction of the time it takes to copy one, and no need for twice the disk space. Of course- the old installation will instantly be missing media files...but presumably when its finished you will not be using it anymore anyways, no?

Begs the question too for larger sites... when dealing with TB+ of data, this is going to need to be able to move files across servers if the file is being copied, is this able to move across a server or do 4x and 5x need to be on the same server?


_________________
Pro JR Hosting, now 50% off!
-100% Guaranteed

"more server and network power than any host, dedicated to your jamroom site"
Back to top
Paul
Jamroom Team


Joined: 20 Aug 2003
Posts: 5341
Location: Nottingham, UK

Posted: 06/17/13 05:16 
Hi DJ
Yes, it does work across servers - that's how I've been doing most of my testing, and if you are doing that, the new server only needs the same disk space that the JR4 site is using.
Yes, moving files instead of copying them would be quicker and would conserve disk space, but it so 'final' Wink
As I said above, things can go wrong and if the original JR4 files have been deleted, there's no second chance.
The process can hang due to server timeouts.
The user might not have set his JR5 quotas up correctly.
The user might change his mind about the prune options.
Whatever, I'm sure its better to give users a second chance instead of saying "sorry, you screwed up, all your data has gone!!".
Now, only when he's happy with the new JR5 site can he then make the decision to delete the old JR4 site.


_________________
Paul Asher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
Dazed



Joined: 22 Mar 2006
Posts: 4147

Posted: 06/17/13 06:10 
I agree - I am on the safe than sorry boat. I think I will end up doing my own backup locally also as a precaution.

Back to top
perfectgamestudios



Joined: 27 Apr 2005
Posts: 698

Posted: 06/17/13 11:14 
another note I think also the video should be run though the conversion via ffmpeg as I see some of the videos do not play and show download button which returns to file not found and checked the jamroom 4 site the file is there and play on jamroom 4

Back to top
Paul
Jamroom Team


Joined: 20 Aug 2003
Posts: 5341
Location: Nottingham, UK

Posted: 06/17/13 11:33 
One thing that is not documented yet with the jrImport module is that as audio and video items are imported, they are queued for conversion. Conversions are a background process and they are kicked off by a page view. This means that only when all imports are complete and you close the modal window does the page refresh and conversions start. Audio conversions are relatively quick - one every few seconds, but video conversions can take a minute or so each. Therefore it could be some time before all conversions are complete and all media plays correctly.
If you periodically refresh the jrImport page it will check and display outstanding conversions in its header.


_________________
Paul Asher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
djmerlyn
Jamroom Ustad


Joined: 18 Dec 2003
Posts: 13497
Location: Behind You

Posted: 06/19/13 15:58 
Is this your script?

jr5/modules/jrCore/router.php


_________________
Pro JR Hosting, now 50% off!
-100% Guaranteed

"more server and network power than any host, dedicated to your jamroom site"
Back to top
Michael
Jamroom Team


Joined: 22 Apr 2008
Posts: 3423
Location: Tokyo

Posted: 06/19/13 16:43 

djmerlyn:
Is this your script?

jr5/modules/jrCore/router.php

router.php is a central part of JR5. It figures out where the user is trying to go.

.htaccess sends people to router.php


_________________
Michael Ussher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
djmerlyn
Jamroom Ustad


Joined: 18 Dec 2003
Posts: 13497
Location: Behind You

Posted: 06/19/13 22:40 
I wonder what is causing it to spool up in to tons of requests and hung sql processes. A user is trying to use the import script, what I see on this end is tons of router.php processes and a saturated sql user.

Looks like this summer is gonna be fun Laughing


_________________
Pro JR Hosting, now 50% off!
-100% Guaranteed

"more server and network power than any host, dedicated to your jamroom site"
Back to top
Michael
Jamroom Team


Joined: 22 Apr 2008
Posts: 3423
Location: Tokyo

Posted: 06/19/13 23:42 

djmerlyn:
I wonder what is causing it to spool up in to tons of requests and hung sql processes. A user is trying to use the import script, what I see on this end is tons of router.php processes and a saturated sql user.

Looks like this summer is gonna be fun Laughing


Everything goes through that script, so its not the script itself thats going to be at the root of the problem. It 'ill be something thats recursive at a guess.

Might be working though, as thats what would happen as a JR5 site sucks in a JR4 data.

You think its broken, or ......? (pauls the expert on this one.)


_________________
Michael Ussher
Jamroom Network Team Member: http://www.jamroom.net
Priority Support: http://www.jamroom.net/Support_Center
Back to top
djmerlyn
Jamroom Ustad


Joined: 18 Dec 2003
Posts: 13497
Location: Behind You

Posted: 06/20/13 00:13 
What it looks like, is that its processing a bunch of queue data all at once, where it should be processing a few things at a time at best I would presume, or at least processing it systematically (one thing after another) instead of a bunch of stuff all at once.


_________________
Pro JR Hosting, now 50% off!
-100% Guaranteed

"more server and network power than any host, dedicated to your jamroom site"
Back to top
Display posts from previous:   
User Support Forum Archive (Read Only)
Jamroom Help

12Next >
 
Solutions
• Social Media Platform
• Social Networking Software
• Musician Website Manager
• Community Builder
Products
• Jamroom Core
• Jamroom Addons
• Jamroom Modules
• Jamroom Marketplace
Support
• Support Forum
• Documentation
• Support Center
• Contact Support
Community
• Community Forum
• Member Sites
• Developers
Company
• About Us
• Contact Us
• Privacy Policy
©2003 - 2010 Talldude Networks, LLC.