my Offline Web App – applicationCaching and manifest files in html5. IT WORKS TOO!

My Need: I tried to make my own iOS app that would take html, scrape the links, get and store the data, change the links and cache a new html file.

OK – as usual – someone already solved that problem (it always happens).

Here are some AWESOME links to help me build my solution – I am SURE you will find them beyond helpful like I did..

Why this app: Something so I can not have to count on my wifi or 3G when reading parent/kid teaching material because we might be camping, or most of the time- JUST out of reach of the wifi and we are lying in bed. 3 year old don’t have patience for you to get in range, start up facebook … again type your group name in only to find that the links on the wall are outdated. I have 3 kids in 3 different age groups. I need something that is JUST GOING TO WORK when I need it to work. And it does – nice.

Design thoughts, steps and resources that you can use too: I want an app that is offline – well html5 has a solution already AND IT WORKS and there are some really smart guys with a lot of good material. Let me bookmark the ones that got me going

  1. installed a wordpress site
  2. got a blank or simple html5 template that I could hack (or build into is what I actually did). I read this book a while back.
  3. in my plugin in I added some code to call my new template and I requested a manifest file with the &manifest=1 GET parameter
    1. this then calls the single.php page which, when it sees the &manifest=1 goes and
    2. calls a new header called header-manifest.php and I=
    3. make the template also redirect to single-manifest.php which together
    4. spits out the manifest headers with[html]Content-type: text/cache-manifest[/html]
    5. Read more about what to spit out with these links
      1. Offline Web Applications – Dive Into HTML5
      2. How to create offline webapps on the iPhone | The CSS Ninja – All things CSS, JavaScript & HTML
      3. This is start to my app which uses what these guys do with javascript so you can SEE or understand what is going on. THESE GUYS ROCK – the client javaScript side needs to be working to monitor progress for your users. Read the comments below and you can see you only need to do an ajax query to the local cached version of your html manifest file (that is cool too) so you don’t have to fetch 3 times from the server
        1. fetch 1 – the page
        2. fetch 2 – the same page but ransacked for all the pdf links and other resources to cache
        3. fetch 3 – not required any longer due to buddy’s comment in the above link – but a pull of the manifest file to do some regex’s and get the # of manifest items so you can see your progress.
      4. To include jQuery – use this link from google
      5. Some stuff from Apple Safari Web Content Guide: Storing Data on the Client
        1. apple  / iOS has some issues with putting the UIWebView and getting the app cache to cache all files. I think I need to increase the cache. For now, I have it opening up Safari and it works great. Here is what might work: Link1, Link2, Link3. These guys seem to have got it to work – I need to get my app out and working.
    6. It works and works great. Click on this link and see the top data (my stuff) loading and the app cache stuff loading live. At the time of writing this, the bottom of the page is quite texty. I need to jQuery it to get it snazzy and graphical.
    7. Turn off the wireless (enter the airplane mode) and keep browsing the links. AMAZING!

      This is the file that gets scraped for the links into a manifest file which is a list of the resources that get cached - offline. Pics too.

      This is the file that gets scraped for the links into a manifest file which is a list of the resources that get cached – offline. Pics too.

The geekery that makes it work - the text version. jQuery to come

The geekery that makes it work – the text version. jQuery to come

HTML5, Manifest files and offline content

So I think I have uncovered something major. In my iOS security app for the preschoolers, I plan to include or make a separate app for looking at curriculum. I WAS going to download to the local app an html page, scrape the links, download those resouces, change the local html page links to local links and – voila- an offline app. Genious … well, it has already been invented with html5 (isn’t it always the way).  I noticed it while looking up how to specify local iOS urls (see part where it says CSS Ninja).

Here are 2 EXCELLENT articles and Q&A’s below them. I am going to try this- it is exactly what I need to make the curriculum offline. Why, if I am reading/reviewing with my kids do I need curriculum offline. I am an at-home and not a circus dad you might say . . . yes, but my wireless router is in the basement and I have a 3 year old with a 3 year old attention span and so do many other parents out there. 3G is still quite slow and I am not sure when Telus here in Calgary is going 4G – besides when the wifi is going in and out and has 1/2 the bars – the phone is so busy establishing which network to use, that it does not download any webpages. Also typing into my facebook app the different sets of curriculum for 3 different kids ages that I require is tedious and annoying – one page that links all of this? Great – just what I need. This offline approach is just what I need I hink and other need as well from small interviews I had with other dads (moms don’t do this techno-learning quite as intensely). Read on over the next while to see how things are going. I am going to serve up the pages and manifest files with php.

The 2nd is beyond a stunning example of someone who knows what he is doing and is referenced by the 1st in the comments as to why things are not going well. Read them in the following order.

First, the content comes from wordpress. This will be a wordpress plugin and I will be using Netbeans PHP for this v7.3 . I have used Netbeans for Java development, but not php – there is a plug in for wordpress plugins.

In short, here is what needs to happen

  1. we need to wrap the html content from wordpress with an html5 wrapper with a manifest directive
  2. serve up the webpage manifest file which is the offline magic file with the proper Content-type: text/cache-manifest  (or put it in a dir with an .htaccess file) and ensure this file NEVER gets cached until it is a mature project.
  3. Serve up a manifest file EACH load (so put the date/time/ip address as a comment in the manifest file) with a reference to every html link, image and pdf in the html content.

Step 0 – Setup

  1. Installed wordpress in a new subdirectory with a new database etc.
  2. Started a new project in netbeans. The wordpress additions require all the credentials of the database etc. These are in the wp-config.php file in the root directory of the new wordpress installation. The DB_HOST is the host of your installation which for wordpress is normally localhost. Since I am developing at home and it is hosted on the web, this needed to change to my server and I had to use the control panel to open up port 3306 (MySQL port) on the web. To only allow my host at home, use ipchicken.com to find your ip.
  3. AHH! Got the error ‘Error establishing a database connection’. Solved it here.
  4. I had a lot of fiddling getting Netbeans to be in a place that made uploading to github easy and would run on my local Mac webserver. (ok – it took another 2 hours to get my Mountain Lion install of apache working too)

Step 1 Get content from WordPress

[html]
<code><!doctype html>
<html lang="en" <mark>manifest="mymanfile.php"</mark>></code>
[/html]

The dreaded ‘Error establishing a database connection’

I am trying to use Netbeans PHP and have a LOCAL copy of WordPress with a remote mysql database.

This was an obscure error which the following php revealed some hints.

[php]

<?php
$db = mysql_connect(‘1.2.3.4’, ‘username’, ‘password’);
if (!$db) echo "connection failed";
else echo "connection succeeded";

echo phpinfo(); ?>

[/php]

Note that the @ is NOT in front of the mysql_connect statement – this means you will see all the errors generated when something goes wrong. What it said did not fix the issue. I do NOT have control over this mySQL install (cannot see or set things in the user table)

It did not like the old password created by the script that made the WordPress installation. So to solve this, after an hour, I finally came acro

Then, our good friends at stack overflow had suggested this ….

[code]
<pre><code>SET SESSION old_passwords=FALSE;
SET PASSWORD = PASSWORD(‘[your password]’);</code></pre>
[/code]

This fixed it.

However, he continues with (I didn’t try this)

[php]<code>define(‘CLIENT_LONG_PASSWORD’, 1);</code>
<pre><em id="__mceDel"><code>mysql_connect(‘[your server]’, ‘[your username]’, ‘[your password]’, false, CLIENT_LONG_PASSWORD);
[/php]

Reachability and SimplePing much improved and useable on iOS

I will do my best to give everyone credit where credit is due – I have looked at so many websites. 

ReachabilityWithSimplePingMuchImprovedIs your app waiting for the network or an online resource and are you ensuring your users know that there is something going on? If you don’t take care of this or give your users an idea of what the phone is doing – or if it is doing anything – the Apple app store apparently could reject your code and some authors out there… have been rejected and you put at the end of the line to resubmit.

So – the app I am making to load webcams for an in-house preschool security system, that is ONLY supposed to work inside the wifi network. Here is a GREAT resource to start with – you can see if www.apple.com is available. BUT … I wanted to know if the IP address 192.168.1.12 (a local LAN ip address) was available – then I know that it is the correct location. The Apple Reachability example AND a MUCH better one from www.ddg.com have a method called reachabilityWithAddress: but it doesn’t work – it says everything is available with ANY I.P. address! Ideally I want to know if the network IP address of the iphone is the same as that network (see WhatsMyIP) but that is not in my camera app yet.

I upgraded www.ddg.com’s code and converted it to ARC compatible code (refactored it) and modernized it a bit for iOS 6.1.

But the biggest thing, is that I added and wrapped Apple’s SamplePing to it as well – since that gets me over the hump. I have a question into the fellow that made Reachability better to see if he had insight to the native reachabilityWithAddress: .

It also has a method to set how many times it Ping’s since SamplePing never stops and how long to retry those subsequent pings. There is a button next to the ping that allows you  to re-ping. It works when the wifi changes, when cellular changes, when VPN changes – all sorts of new information available with a utility to help you code your own app instead of guessing what might be working or not.

You can edit the ip address live and it resets/retries everything again X times – so in fact it is a useful tool for when you go ‘on the road’ to see what flags etc. and conditions your new app sees. In my case, I need to know if the wifi is live, and if I can ping a certain IP address. Then I can let my users know with an activity indicator, what is going on and why the camera’s do not load – because most of the time they should not load. When your at the customer’s location, the cameras should load. Outside or using cellular 3G/4G or cellular data – it should tell them that things will not work  and how to obtain the guest ID of the customer’s wifi network.

Interesting points and links I used along the way that are not already listed:

  • SimplePing locks the phone for up to a minute if you continuously ping without waiting for a response. Let it time out before you retry AND run that code in a different thread. New concepts exist that we can take advantage of like closures in Groovy.  Here is “Grand Central Dispatch” tutorial #1 and #2 that I used
  • Githib and XCode – I have little hair to spend on this. Spend the $$ and download Tower for Mac. Ahhh – thats better.
  • apple’s sockets and timeouts
  • another good article about reachability that helped me understand

Here is the code on git. Send me a line if you end up re-using it – it is nice to know who and where on the Globe one’s stuff gets to!

iPhone app for family connection and preschool security camera viewer

The human request was to see all the video camera feeds from the preschool classroooms on an iphone.

Great: This is an app for a non-profit organization that has preschool classes and other ages above this. The intent is that parents can take the material and continue mentoring their kids ALONG with the organization. This organization is a church – so that is why the family focus.

Step 1 – Make it a tabbed app. Since the video portion is not going to be the only use for the app lets make it a tabbed app.

Step 2 – figure out the video feeds. These are foscam FW8919W cameras (I can’t believe they sell for <$90 – these are beyond full featured cameras). Here is the link to figure all this out from foscam.  It turns out that the most memory efficient is the MJPEG route with an UIImage item. Googling revealed this guy who has a side-by-side test of a web view vs. image view on the iphone and helped me figure this out. I used his class he has in that app – thank you.

Step 3 – using storyboarding and a UI Scroll View with paging enabled, the app loads in a jiffy. Now I need to unhard code it (arrgg) … but that is done as well.  That article also outlines how to do a page controller. Don’t forget to set the .numberOfPages property which that article is missing. I also changed a label above the movie so you know what room in english you are looking at.

Step4 – add a web browser UIWebView to the 2nd storyboard and a back button to make it go back

Step5 – make a webpage and link it into step 4 (this took the longest ironically). The size is 320 by 420px for an iPhone

Step6 – add in some user preferences so that when the user comes back – it remembers the last camera it was viewing.

Step7 – test it all. Finally after about 6 hours it all works.

Testing on the real iPhone

Now I upgraded my iPhone to version 6 and my XCode does go to version 6 because I am still running Leopard for performance reasons. I guess I will have to upgrade to Lion now (arrg)

Now I noticed things that I could not on the simulator

  • I cannot scroll my PDFs in the UIWebview
  • The system should tell me when I am in an environment where it won’t work (it only should work in the intranet – the over the internet)
  • a few other details like turning off cameras when the paged view of the cameras was not on that particular camera
  • camera should not rotate. This was a little tricky. I tried to edit the orientations method- (BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)interfaceOrientationbut to no avail. Edit the AppName-Info.plist and delete the rows (or plist entries) for all devices (ipad and iPhone that are not button bottom.

Finishing Touches

  • made all icons as per this article and dragged them from the finder into the app’s main page in xcode and the summary tab
  • reconnect everything in the iPad story board (re-learn, re-remember all the connections you did)

Here are some quick links for now so that everyone can follow what I had to do to get this working. When I have time, I will back fill the information in:

setting up Time Machine to see a network drive instead of a local drive.

To allow macs to see other drives other than local ones (see link below), you basically

  • edit a setting on your mac
  • create a sparse bundle using some unix commands – make it at least 3 times larger than your expected data to be backed up
  • MOVE that sparse bundle to a remote machine and connect to that remote directory – then tell time machine to start looking for hard drives.

 

Here – just do what Stephan says and read with understanding and he leads you through the bullet points above. This stuff is written so you CAN understand what is going on. I learned this stuff long before he simplified it – so thank you.

Solved: Fonts on a Mac for Microsoft office – why are they not updating? Where are they?

Macs are not perfect even though one’s productivity is much higher with one. When things go wrong – they can sometimes really go wrong.

One customer had issues with a critical branding font that installs fine on everyone else’s Macbook but not theirs. I now know more about fonts that I ever wanted to know especially how Microsoft fonts fit into the Mac picture. I researched dozens of websites each adding a piece to the puzzle. Here we go …

  • There are 4 font stores on your Mac
    • user fonts stored in /Users/youraccount/Library/Fonts
    • computer wide (all accounts) fonts in /Library/Fonts
    • system fonts in /System/Library/Fonts (never ever touch these)
    • Microsoft Office fonts in /Library/Fonts/Microsoft (ahhh I see)
  • ONLY TTF fonts work for Office – or so MS claims
  • To install fonts for MS Office 2011 – don’t double click them – this installs them naturally in the user fonts
    • instead – start up Font Book (use the spotlight or magnifying glass to find it quick in the upper right of your Mac)
    • drag them from your Finder onto Computer (under the Collection section at the left of the Font Book app)
  • A reboot triggers the Mac Font store to sync with Office. Don’t forget to reboot before ripping your hair out
  • Adding fonts to the user fonts will never show up in Microsoft Office products
  • Don’t assume that because a font works in lets say Word, that it will appear in Excel – it might once the cache catches up
  • Microsoft font cache file can be delete so it will force a refresh – but it can be in 2 places – check both. Microsoft moved it for Office 2011 for Mac for some computers different than others
    • Lion? goto finder and hold the option key and …
    • click the menu Go->Library or type in a folder /Library by choosing the Go->Go to Folder option then navigate eventually to /Library/Preferences/Microsoft/Office 2011
      • not there? Microsoft moved them in later releases of Office 2011 to /Library/Application Support/Microsoft/Preferences/Office 2011 – even MS’ articles are incorrect!
      • also you might need to look in /Users/yourname/Library/Application Support/Microsoft/Preferences/Office 2011
    • when you re-start work or Excel you will now see a task completing to rebuild the cache files and if you observe the location above you will see new cache files
  • Here is a link to completely remove Office on a mac and is the final puzzle piece that allowed me to solve this riddle http://support.microsoft.com/kb/2398768

Many thanks to these sites:

How to send emails from iOS devices from the one you want

The service providers and even the Applestore do not know the simple art of setting up an email account NOT as an iCloud account or provider account. It is simple to do knowing a few basics of what you can do. A lot of us have our own domain names or preferred email accounts that we want to send “from” and appear as to others. It keeps things cohesive and flexible to change if that service provider is not a preferred one. I do this for all my customers and it takes about a minute longer.

First, use only the services you need to to get the job done. If you are setting up Gmail – then do NOT click on new account and choose the Gmail Type – choose other and type in the details the long way. THEN turn off every service except email for this new account and turn OFF email for the iCloud but leave all the other services like notes, etc. See? Substitute only the service you replacing – not all the services with this new other type account.

  • your name as you want it to appear
  • your email address – any email address
  • incoming server is imap.gmail.com
    • type in your full username and password as directed by the service you want to use – so google the settings for … telus or shaw or gmail.
  • outgoing server is smtp.gmail.com
    • again you have to do some googling for settings but it is usually the same as your incoming server
  • if you want you can use an existing smtp setup that you know already works. I have set up a special email account for outgoing mail only – so I would use that smtp server on ALL my email accounts set up on my iDevice once I have the basics setup

When you send mail make sure it works over wifi and 3G/4G so that you won’t get to a friends house and cannot send things until you get back to home or work.

Make sure your ‘default account’ for sending is the one you want. When you add new accounts, the new one might take over and you might not want this to happen.

Would you like ONLY calendar info then from gmail? Simply choose to add a gmail the simple way – but because we set up email independently using the new other account type – be sure to turn OFF mail on the gmail account.

Bonita BPM opensource from Bonitasoft – what is it?

Bonita Open Source (BOS) is a system to allow an organization that is local or across the globe to work a process from start to it’s finish and know where in the process one is. You can start one or dozens of the same one or different ones. As a friend of mine said – you are really programming people – project management without the overhead of training or making costly mistakes – this system leads them through the whole thing.

If you leave process sitting at your desk for the weekend – it will be waiting there on Monday morning. If the process is stale, it could kick off another process to alert others that there is a hold up. Since I had the pleasure of sharing my first Bonita experience with a good friend of mine, I will redirect you to his site for a more business level discussion of what Bonita OS is. I was the implementer of most of the processes.

Platform independant

It is web based and therefore platform independent and simply connect human processes and be an alert mechanism between people or it can complete complex processes that are already in place. It is programmed using Java and Groovy and can connect ANYTHING it seems together in an automated fashion

Setup of the system and users

It is not the easiest too to get under way but when it comes to getting processes kicked off, getting snibbets of info from person 1, then moving on to person 2 to get more info and keeping track of where a process is in the grand flow, I have never seen anything more complete.

Setting up users, roles, hand-off and moving processes forward in a managed way is quite easy with Bonita Open Source (BOS) from Bonitasoft. They seemed to have thought of it all as real people get stuck, make mistakes, go on holidays etc. and the show must go on. Bonita accounts for all that.

We started with BOS 5.9 and it includes a designer suite and published modules get uploaded to a server version in a VERY controlled manner. There will be no cowboy programming techniques allowed with this system which is EXCELLENT for you customers or end users.

Programming

The basic flow is programmed with standard BPMN notation but with a programming flair. This article by a colleague is clearer than I can tell you so click here. Communicating design with the customer is therefore a breeze since the flow can easily be seen visually.

The documentation is perhaps complete but the whole picture is only complete with the documentation at Bonitasoft, their forums some videos, and lastly trying to get it to work yourself with small experiments. I was fortunate enough to have a friend at geekinc.ca to share the learning curve with and we could share ideas. I would get good at integrating custom extensions while he proceeded to get good at making once process flow talk to another thereby keeping things modular.

Their source code is available and published documentation of all the api’s (classes) available. The answers are there – there is just no examples of how one feature interacts with another. This is better than one company I worked for which had an api as big as this and there was no documentation except walking into the 17 year old veteran’s office and he and I would step through the code and get an explanation of the features. If there was only someone who wrote a book … oh that is the eventual intent of this blog is to cover that gap in the meantime.

Integrate just about anything

At first, you can cobble together a process with just human pseudo tasks that bounce to the correct people. Then one by one, you replace one human task with a sub-process which contains tasks that in of themselves are programming but could also be what is called a connector or plug in. Since the system is based upon Java and Groovy, download Netbeans and start creating classes. Connectors are themselves classes that can calll custom classes. We have integrated a system automatic PDF generation using PDF-tk which is a command like tool by interfacing it with a simple piece of Java code packaged as a jar. Other simple examples is an “Excel” database lookup. A matrix was provided to a Java jar which processed an excel file and gave back a list of strings from that Excel file. Anything can be integrated together since Java is cross platform AND works across machines quite easily. The hard part about Bonita is finding out all it can do since the possibilities seem endless like Sharepoint, Microsoft, MySQL, Oracle, groovy, google and hundreds of other connectors already exist let alone the ones you dream up.

Moving from design to production

The final implementation for end use is on a server installed version of Bonita. Integrating design iterations in must be done in a version controlled way which is slower, but better for the customer’s expectations and the end users experience.

How far does the Open Source take us – what if we want to pay?

This all depends on who is paying for things. The open source has its limitations like inter-webform validations and document uploading into the process. Many things are made easier with a paid subscription and there are 3 levels you can purchase from Bonitasoft. You get a few sample templates to ease Bonitasoft’s very sparse documentation.

WordPress – where to start with plugins

I am now a fan of WordPress. It is not that it is the best for myself or my customers but it is the penetration into the market and how well it gets along with its friends like Google, Facebook and so many other web services. There are more plugins that do the same thing, there is surely one out there that meet your needs.

Integrating and customizing themes is quite easy once you understand the file structure as well. Overall it is effective and therefore inexpensive to have running. I first learned it from a good friend who shared what he knew at geekinc.ca which rocketed me from an intense Joomla knowledge into the next wave of content management systems. As I use it, my customers ask questions etc., I narrowed it down to this list.

Standard Plugins I would recommend any site to employ would be…

  • Add link to Facebook
  • Captcha seems to work quite well although I made some changes so it uses images – the company that made it, doesn’t want to update it, so I only updated it so far. When the spam continues, I will finish it. Seems to have stopped spam so if you want it, it works for now.
  • [optional] My Twitter Widget does what it claims – puts your tweets on the side of your website. No I am not using it because I don’t tweet (yet). With a little CSS magic it cleans up just the way you require.
  • [depreciated in my use – see Hupso] WP to Twitter Widget works well
  • Hupso Share Buttons for Twitter, Facebook & Google+  I am using this instead of WP to Twitter because I can just click these buttons to self-tweet and self-post to facebook AND MUCH MORE! These are all the little icons under my posts
  • Akismet – you get what you pay for – this one works well to stop spam. I am not using it because I am too cheap – but it does what it claims for my customers
  • Exclude Pages from Navigation
  • FD Feedburner Plugin
  • Google Analytics for WordPress
  • Google XML Sitemaps
  • [optional] Login with Ajax
  • [optional] podPress
  • WordPress SEO– the one by Yoast
  • [optional] Contact Us This one is good and very configurable to make a contact us form. I am using Dropifi as it is free to set up an account.
  • [optional] Dropifi – see the ContactUs flyout on my site … enough said.
  • Image Paste – you can just simply “paste” images in – too cool. The only drawback is that the images are embedded in the blog most – not the media directory. If your editor is having a bad day. .. not good – no more pics
  • [optional for coding geeks] SyntaxHighlighter Evolved
  • Insert Html Snippet just install this – why? Because if you have paypal buttons or any other HTML that needs to be “just-so” then your text editor will not format it incorrectly since it is an abstraction. You will need it at some point, so just insert it and learn how to use it.
  • [optional] Smart Youtube If you have a youtube account or post videos – just paste your link and change http to httpv and … by magic – the video just is there and looks great. How simple is that?