This repository was archived by the owner on Oct 22, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathrss.xml
More file actions
5043 lines (3978 loc) · 663 KB
/
rss.xml
File metadata and controls
5043 lines (3978 loc) · 663 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Grok Interactive</title>
<link>http://www.grok-interactive.com</link>
<atom:link href="http://www.grok-interactive.com/rss.xml" rel="self" type="application/rss+xml" />
<description>Grok Interactive's web developers create outstanding applications</description>
<language>en-us</language>
<pubDate>Tue, 03 Nov 2015 15:22:20 -0500</pubDate>
<lastBuildDate>Tue, 03 Nov 2015 15:22:20 -0500</lastBuildDate>
<item>
<title>09 - Security Basics for the Consumer and Developer</title>
<link>http://www.grok-interactive.com/podcast/security-basics-for-the-consumer-and-developer</link>
<pubDate>Tue, 03 Nov 2015 00:00:00 -0500</pubDate>
<author> ()</author>
<guid>http://www.grok-interactive.com/podcast/security-basics-for-the-consumer-and-developer</guid>
<description><p>Listen along as Josh, Paul and Omar of def-Logix.com discuss cyber security and research development.</p>
<p>Paul and Omar tap their experience to give a short talk about what to do, what not to do as well as some common mistakes and misconceptions in the web space regarding security. </p>
</description>
</item>
<item>
<title>08 - Test Driven Development</title>
<link>http://www.grok-interactive.com/podcast/test-driven-development</link>
<pubDate>Tue, 27 Oct 2015 00:00:00 -0400</pubDate>
<author> ()</author>
<guid>http://www.grok-interactive.com/podcast/test-driven-development</guid>
<description><p>Listen along as Josh, Anton and Christopher discuss one of our favorite topics; Test driven development. </p>
<p>Test-driven development (TDD) is a software development process that relies on the repetition of a very short development cycle: first the developer writes an (initially failing) automated test case that defines a desired improvement or new function, then produces the minimum amount of code to pass that test, and finally refactors the new code to acceptable standards. Kent Beck, who is credited with having developed or &#39;rediscovered&#39; the technique, stated in 2003 that TDD encourages simple designs and inspires confidence.</p>
</description>
</item>
<item>
<title>Upload a Jekyll Site to AWS with Continuous Deployment</title>
<link>http://www.grok-interactive.com/blog/deploying-jekyll-site-to-aws/</link>
<pubDate>Thu, 22 Oct 2015 00:00:00 -0400</pubDate>
<author>joshf@grok-interactive.com (Josh Freeman)</author>
<guid>http://www.grok-interactive.com/blog/deploying-jekyll-site-to-aws</guid>
<description><p>When Grok builds applications, we strive to follow the Agile approach to software development. Continuous Integration and Continous Delivery is paramount to our business. We&#39;re able to deploy changes to our clients&#39; websites dozens of times per day (if we choose). However, one place we haven&#39;t used CI/CD is this blog!</p>
<h2>Why Not?</h2>
<p>When this site was first built, we used Github Pages to host our website. It&#39;s certainly not the best approach, but deployment was fast and management was simple If you have a Jekyll site (like this one), pushing a branch to <strong>gh-page</strong> will build and serve the static site.</p>
<p>Jekyll blog authors always tend to need more flexibility, so naturally we install plugins! Only there&#39;s a problem–Github disables Jekyll builds that use plugins. If we want to continue using Github to host our site, we now require <em>another</em> repository to host the site. We build our Jekyll site locally, copy the files to the new static repository, commit, and push to Github.</p>
<p>We have traded the convenience of continuous deployment for the flexibility of Jekyll plugins. We deploy changes several times per week--sometimes several times per day. It works, but it&#39;s cumbersome, and error prone. </p>
<h2>The Setup: Amazon Web Services (AWS) &amp; Codeship</h2>
<p>Hosting a static website is about the simplest thing you can do on the internet–web servers were literally designed for this single purpose. But we can avoid having to manage a server all together. Enter Amazon Web Services and Codeship.</p>
<p>With AWS, we get extraordinarily cheap website delivery and resilience. With Codeship, we&#39;re given a listener that will deploy changes on Github to AWS.</p>
<p>Let&#39;s put it all together. </p>
<h3>File Storage: Amazon Simple Storage Service (S3)</h3>
<p>S3 is the file storage service we&#39;ll use to host our website. It&#39;s simple enough to think of it as a filesystem that we&#39;ll expose to a URL.</p>
<ul>
<li>Create a bucket. Ours is named <code>www.grok-interactive.com</code>.</li>
<li>Under <strong>Static Website Hosting</strong>, choose <em>Enable website hosting</em> and provide the index document (probably <code>index.html</code>) and the error document (probably <code>404.html</code>)</li>
<li>Under Permissions, edit the bucket policy:</li>
</ul>
<div class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span>
<span class="nt">&quot;Version&quot;</span><span class="p">:</span> <span class="s2">&quot;2012-10-17&quot;</span><span class="p">,</span>
<span class="nt">&quot;Statement&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">&quot;Sid&quot;</span><span class="p">:</span> <span class="s2">&quot;AddPerm&quot;</span><span class="p">,</span>
<span class="nt">&quot;Effect&quot;</span><span class="p">:</span> <span class="s2">&quot;Allow&quot;</span><span class="p">,</span>
<span class="nt">&quot;Principal&quot;</span><span class="p">:</span> <span class="s2">&quot;*&quot;</span><span class="p">,</span>
<span class="nt">&quot;Action&quot;</span><span class="p">:</span> <span class="s2">&quot;s3:GetObject&quot;</span><span class="p">,</span>
<span class="nt">&quot;Resource&quot;</span><span class="p">:</span> <span class="s2">&quot;arn:aws:s3:::www.grok-interactive.com/*&quot;</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div>
<p>We now have a place to store our website. Next we need to setup the permissions to automate its management.</p>
<h3>Account Management: Amazon Identity and Access Management (IAM)</h3>
<p>IAM is the user management service we&#39;ll use to permit the command-line deployment of our files to S3. We will create a user and provide policies to enable access to our S3 bucket. </p>
<ul>
<li>Create a user and copy the <em>Access Key ID</em> and <em>Secret Access Key</em>.</li>
<li>Create an In-Line User Policy to enable full access to the S3 bucket with the following:</li>
</ul>
<div class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span>
<span class="nt">&quot;Statement&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">&quot;Action&quot;</span><span class="p">:</span> <span class="s2">&quot;s3:*&quot;</span><span class="p">,</span>
<span class="nt">&quot;Effect&quot;</span><span class="p">:</span> <span class="s2">&quot;Allow&quot;</span><span class="p">,</span>
<span class="nt">&quot;Resource&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;arn:aws:s3:::www.grok-interactive.com&quot;</span><span class="p">,</span>
<span class="s2">&quot;arn:aws:s3:::www.grok-interactive.com/*&quot;</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div>
<ul>
<li>Create another In-Line User Policy to remove S3 bucket creation/deletion with the following: </li>
</ul>
<div class="highlight"><pre><code class="language-json" data-lang="json"><span class="p">{</span>
<span class="nt">&quot;Version&quot;</span><span class="p">:</span> <span class="s2">&quot;2012-10-17&quot;</span><span class="p">,</span>
<span class="nt">&quot;Statement&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="p">{</span>
<span class="nt">&quot;Effect&quot;</span><span class="p">:</span> <span class="s2">&quot;Deny&quot;</span><span class="p">,</span>
<span class="nt">&quot;Action&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;s3:CreateBucket&quot;</span><span class="p">,</span>
<span class="s2">&quot;s3:DeleteBucket&quot;</span><span class="p">,</span>
<span class="s2">&quot;s3:DeleteBucketPolicy&quot;</span><span class="p">,</span>
<span class="s2">&quot;s3:DeleteBucketWebsite&quot;</span><span class="p">,</span>
<span class="s2">&quot;s3:DeleteObjectVersion&quot;</span>
<span class="p">],</span>
<span class="nt">&quot;Resource&quot;</span><span class="p">:</span> <span class="p">[</span>
<span class="s2">&quot;arn:aws:s3:::www.grok-interactive.com&quot;</span><span class="p">,</span>
<span class="s2">&quot;arn:aws:s3:::www.grok-interactive.com/*&quot;</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
</code></pre></div>
<p>Validate the policies and simulate to ensure it works like we expect. </p>
<p><img src="/images/2015/10/policy-simulator.gif" alt="Amazon IAM Policy Simulation"></p>
<h3>Continuous Integration &amp; Deployment: Codeship</h3>
<p>Codeship is the CI/CD service we&#39;ll use to automatically deploy changes to the website. Without tests, we get continuous integration effectively free–our only failure would be if Jekyll fails to build. However, with Codeship, we can build our site within their virtual machine and then push the built site to S3.</p>
<ul>
<li>Create a project and add connect your Github repository</li>
<li>Under <strong>Project Settings: Test</strong>, provide the following setup commands:</li>
</ul>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">rvm use 2.2.3 --install
pip install awscli
bundle install
jekyll build
</code></pre></div>
<ul>
<li>Under <strong>Project Settings: Deployment</strong>, add a new pipeline that only runs on your <strong>master</strong> branch that will execute the following:</li>
</ul>
<div class="highlight"><pre><code class="language-sh" data-lang="sh">aws s3 sync _site s3://www.grok-interactive.com --delete
</code></pre></div>
<ul>
<li>Under <strong>Project Settings: Environment</strong>, provide the following from your IAM account:
<ul>
<li>AWS_ACCESS_KEY_ID</li>
<li>AWS_SECRET_ACCESS_KEY</li>
<li>AWS_DEFAULT_REGION</li>
</ul></li>
</ul>
<p>At this point, it&#39;s a good idea to check that commits to your <strong>master</strong> branch on Github will trigger a build on <strong>Codeship</strong> and deploy the static files to S3.</p>
<p>Our S3 endpoint is <code>www.grok-interactive.com.s3-website-us-west-2.amazonaws.com</code>, and you can find yours under the <strong>Static Website Hosting</strong> panel in your S3 console. </p>
<p>Because our site has a few plugins, the exact commands are slightly different, but the intent is the same. If you ever want to redeploy, or test you configuration, you can re-run a build.</p>
<p><img src="/images/2015/10/codeship.gif" alt="Codeship Integration Build"></p>
<h3>Content Deliver Network: Amazon CloudFront</h3>
<p>CloudFront is our CDN. When Github has problems (like suffering DDOS attacks from China, for instance), we tend to suffer downtime as well. By using CloudFront, we greatly reduce the latency for file downloads and improve our website&#39;s resilience.</p>
<ul>
<li>Create a <strong><em>web</em> distribution</strong></li>
<li>Set the origin domain name to your S3 Static Website URL. <strong>Do NOT use the auto-dropdown, paste the URL</strong>. The origin path should autopopulate with <code>Custom-</code> prepended to your URL.<br></li>
</ul>
<p><img src="/images/2015/10/cloudfront-origin-settings.png" alt="CloudFront Origin Settings"></p>
<ul>
<li>Set your <strong>Default Cache Behavior Settings</strong>. Here&#39;s ours:</li>
</ul>
<p><img src="/images/2015/10/cloudfront-cache-settings.png" alt="CloudFront Origin Settings"></p>
<ul>
<li>Set your <strong>Distribution Settings</strong>. Here&#39;s ours: </li>
</ul>
<p><img src="/images/2015/10/cloudfront-distribution-settings.png" alt="CloudFront Origin Settings"></p>
<h3>DNS: Amazon Route 53</h3>
<p>Route 53 is the service we&#39;ll use to enable Amazon to internally route requests to our website to the correct services.</p>
<ul>
<li>Create a hosted zone</li>
<li>Add an Alias record for <code>www.grok-interactive.com</code> to point to the S3 Website Endpoint</li>
<li>Add an Alias record for <code>grok-interactive.com</code> to point to the S3 Website Endpoint</li>
<li>Migrate any remaining records from your other registrar/DNS provider to Route 53. Execututing <code>host -a grok-interactive.com</code> is <em>really</em> useful for this. </li>
</ul>
<h3>Update Your Nameservers</h3>
<p>Go to your Domain registrar (Domain.com, GoDaddy, Namecheap, etc.) and update your nameservers to the nameservers provided in Route 53. Now that your domain points to Route 53, AWS will know where to send each request. Each registrar has a different control panel to do this, so I&#39;ll refer you to their documentation.</p>
<p>Remember, that updating DNS records can take up to several hours to propagate through the internet.</p>
<h2>Conclusion</h2>
<p>We have taken our static website from a manual deployment on a single node and converted into a continuously deliverable website powered with a CDN that provides low latency and high resiliency, for less than a dollar per month. Great return for a few hours time and a few bucks a year.</p>
<blockquote>
<ol class="abcd">
<li><strong>A.</strong> Always</li>
<li><strong>B.</strong> Be</li>
<li><strong>C.</strong> Continuously</li>
<li><strong>D.</strong> Deploying</li>
</ol>
</blockquote>
<h2>Thanks and Resources</h2>
<p>There are few original thoughts. Thanks for all the help!</p>
<ul>
<li><a href="http://danielwhyte.com/app/design/2014/10/05/creating-a-jekyll-s3-server.html">Create a jekyll blog &amp; host it on amazon s3 by Daniel Whyte</a></li>
<li><a href="http://www.michaelgallego.fr/blog/2013/08/27/static-website-on-s3-cloudfront-and-route-53-the-right-way/">Static website on S3, CloudFront and Route 53, the right way! by Michaël Gallego</a></li>
<li><a href="http://www.leveluplunch.com/blog/2014/09/29/amazon-cloudfront-s3-subfolders-default-index/">Setting up cloudfront with S3</a>
<style>
.abcd {
list-style-type: none;
margin: 16px 0;
font-size: 20px;
}</li>
</ul>
<p>.abcd strong {
color: #f68423;
}
</style></p>
</description>
</item>
<item>
<title>Better search engine results with Schema.org</title>
<link>http://www.grok-interactive.com/blog/better-search-engine-results-with-schema-org/</link>
<pubDate>Thu, 22 Oct 2015 00:00:00 -0400</pubDate>
<author>jasone@grok-interactive.com (Jason Ellis)</author>
<guid>http://www.grok-interactive.com/blog/better-search-engine-results-with-schema-org</guid>
<description><p>Grok Interactive in and of itself is an ever evolving going concern. We are always iterating on new ideas and trying new things. We have the luxury of being agile in a nimble sort of way which allows us to try new things or pivot on an idea in short order. </p>
<p>So when we started working on a <a href="http://www.grok-interactive.com/podcast/">podcast</a> we wanted to get the word out quickly via our blog what it was we were doing and that we thought everyone should give it a <a href="http://www.grok-interactive.com/podcast/">listen</a>!</p>
<p>The issue is we noticed our search engine results page click through rate (SERP/CTR) was kinda poor. Our SERP presence was poor to say the least so our CTR suffered.</p>
<p>We needed to fix this and started to explore ways of doing so; Enter <a href="http://schema.org">Schema.org</a>.</p>
<h4>Schema.org</h4>
<p>Back in June of 2011 Schema.org was launched and pretty much immediately both Google and Bing! implemented rich snippets on their SERP using the metadata that could be gleaned from the Schema.org data added to the searched pages.</p>
<p>Schema.org gives you the developer a handy ability to convey a very large amount of information to search engines about what exactly an element represents and is implementable with very little effort. </p>
<p>There are a lot of different tags for various content types which in turn become &quot;rich snippets&quot; on your favorite search engine results; music, events, blog posts, products, recipes..etc. Anything you can think of can likely be expressed and better defined by Schema.org microdata and microformats. And doing so will only help your rankings.</p>
<h4>Microdata</h4>
<p>Schema.org provides you with a &quot;microdata language&quot; and a &quot;schema vocabulary&quot; that while not a true language in and of itself, is a way to make your HTML tags more machine readable which the major search engines then use to better optimize and sort your data for display purposes.</p>
<p>Your various webpages are easy for people to understand when they read them but search engines aren’t people and as such have a very limited and often broken understanding of what is actually being discussed on those pages. We need to help the machine by adding some additional information to the tags on the page in order to help the search engines better understand what the content you’re trying to display actually is in a relevant and useful way.</p>
<p>For example - Let’s say you’re creating a blog post. The HTML might look something like this:</p>
<div class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;article&gt;</span>
<span class="nt">&lt;header&gt;</span>
<span class="nt">&lt;h1&gt;</span>Schema.org is pretty cool<span class="nt">&lt;/h1&gt;</span>
<span class="nt">&lt;address&gt;</span>
by <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">&quot;mailto:jason.ellis@grok.rocks&quot;</span><span class="nt">&gt;</span>Jason Ellis<span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/address&gt;</span>
<span class="nt">&lt;/header&gt;</span>
<span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">&quot;content&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;/div&gt;</span>
<span class="nt">&lt;/article&gt;</span>
</code></pre></div>
<p>This is dated and while modern search engines would read it and you would show up in the search results, your position in those results would be directly affected by the lack of Schema.org microdata. The simple fact of the matter is, the search engine back end logic can only do so much to intuit what this type of data is and how it should be displayed. </p>
<p>We need to give it some more information in order to convey exactly what it is we are trying to accomplish ~ lets help the search engine a bit:</p>
<div class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;article</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">&quot;http://schema.org/BlogPosting&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;header&gt;</span>
<span class="nt">&lt;h1</span> <span class="na">itemprop=</span><span class="s">&quot;headline&quot;</span><span class="nt">&gt;</span>Schema.org is pretty cool<span class="nt">&lt;/h1&gt;</span>
<span class="nt">&lt;address&gt;</span>
by <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">&quot;mailto:jason.ellis@grok.rocks&quot;</span><span class="nt">&gt;</span>Jason Ellis<span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/address&gt;</span>
<span class="nt">&lt;/header&gt;</span>
<span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">&quot;content&quot;</span> <span class="na">itemprop=</span><span class="s">&quot;articleBody&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;/div&gt;</span>
<span class="nt">&lt;/article&gt;</span>
</code></pre></div>
<p>As you can see the tags themselves haven’t changed. We are still using the standard HTML5 elements and haven’t moved them around or anything. But we have added some Schema.org Microdata and schema vocabulary.</p>
<p>Let’s look at each one individually:</p>
<h4>itemscope</h4>
<div class="highlight"><pre><code class="language-html" data-lang="html"> <span class="nt">&lt;article</span> <span class="na">_itemscope_</span> <span class="na">itemtype=</span><span class="s">&quot;http://schema.org/BlogPosting&quot;</span><span class="nt">&gt;</span>
</code></pre></div>
<p>The &quot;itemscope&quot; microdata keyword added to a specific element tells the reader that we are going to be discussing a block of data contained within this specific tag. It’s your &quot;Begin here&quot; attribute which gives the system a starting point. There can be many on a page.</p>
<h4>itemtype</h4>
<div class="highlight"><pre><code class="language-html" data-lang="html"> <span class="nt">&lt;article</span> <span class="na">itemscope</span> <span class="na">_itemtype=</span><span class="s">&quot;http://schema.org/BlogPosting&quot;</span><span class="na">_</span><span class="nt">&gt;</span>
</code></pre></div>
<p>The &quot;itemtype&quot; keyword tell us is specifically which schema vocabulary we are talking about by linking to a schema vocabulary on Schema.org. You can (and it’s a good idea to) open that page up in your browser and read exactly what key/value pairs it can accept. More over there is generally a good example at the bottom of the page how your content should look once you have all the microdata in place.</p>
<p>In this case we are saying everything inside this itemscope is the itemtype &quot;BlogPosting&quot;. </p>
<h4>itemprop</h4>
<div class="highlight"><pre><code class="language-html" data-lang="html"> <span class="nt">&lt;h1</span> <span class="na">_itemprop=</span><span class="s">&quot;headline&quot;</span><span class="na">_</span><span class="nt">&gt;</span>Schema.org is pretty cool<span class="nt">&lt;/h1&gt;</span>
</code></pre></div>
<p>Once you have opened up the schema.org page on the itemtype you’re trying to better define you’ll see that there are a lot of different item properites that can be defined. </p>
<p>Each itemprop key/value pair directly defines what this given tag and it’s content is and how it relates to the information contained within.</p>
<p>Item props have an expected value type as defined by the Schema.org schema vocabulary page. Some can be text, some have to be specific types of other Schema.org data. </p>
<p>There is actually quite a lot of data that you can add to your pages which will get extremely specific about defining all the things according to Schema.org. How far you go down that particular rabbit hole is up to you but the more defined you get, the better it helps the various engines with SERP/CTR and SEO. So it’s a pretty good idea to make it happen.</p>
<p>Taking it a bit deeper, lets define the author;</p>
<h4>Author</h4>
<div class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt">&lt;article</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">&quot;http://schema.org/BlogPosting&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;header&gt;</span>
<span class="nt">&lt;h1</span> <span class="na">itemprop=</span><span class="s">&quot;headline&quot;</span><span class="nt">&gt;</span>Schema.org is pretty cool<span class="nt">&lt;/h1&gt;</span>
<span class="nt">&lt;address</span> <span class="na">itemprop=</span><span class="s">&quot;author&quot;</span> <span class="na">itemscope</span> <span class="na">itemtype=</span><span class="s">&quot;http://schema.org/Person&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;meta</span> <span class="na">itemprop=</span><span class="s">&quot;email&quot;</span> <span class="na">content=</span><span class="s">&quot;jason.ellis@grok.rocks&quot;</span><span class="nt">&gt;</span>
by <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">&quot;mailto:jason.ellis@grok.rocks&quot;</span> <span class="na">itemprop=</span><span class="s">&quot;name&quot;</span><span class="nt">&gt;</span>Jason Ellis<span class="nt">&lt;/a&gt;</span>
<span class="nt">&lt;/address&gt;</span>
<span class="nt">&lt;/header&gt;</span>
<span class="nt">&lt;div</span> <span class="na">class=</span><span class="s">&quot;content&quot;</span> <span class="na">itemprop=</span><span class="s">&quot;articleBody&quot;</span><span class="nt">&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;p&gt;</span>.... article content ....<span class="nt">&lt;/p&gt;</span>
<span class="nt">&lt;/div&gt;</span>
<span class="nt">&lt;/article&gt;</span>
</code></pre></div>
<p>Here you can see I have told the system that the <code>address</code> tag is a new item scope and everything inside of it relates to the schema.org item type of Person. </p>
<p>I then added a meta tag with the itemprop of email to define the email of the author since the anchor tag which is a mailto actually defines the users name. There really is no limit to how deep you can go. Give it a try and see if your search results don&#39;t improve. </p>
<h4>Testing</h4>
<p>As with all things you need a way to test. I highly recommend using Google&#39;s testing tool, <a href="https://developers.google.com/structured-data/testing-tool/">https://developers.google.com/structured-data/testing-tool/</a></p>
</description>
</item>
<item>
<title>07 - Ryan Orsinger Interview</title>
<link>http://www.grok-interactive.com/podcast/ryan-orsinger-interview</link>
<pubDate>Tue, 20 Oct 2015 00:00:00 -0400</pubDate>
<author> ()</author>
<guid>http://www.grok-interactive.com/podcast/ryan-orsinger-interview</guid>
<description><p>Listen along as we interview and generally cat with a long time friend of Grok Interactive, a former employee of same and current Codeup Instructor; Ryan Orsinger. </p>
<p>Ryan discusses his experiences as a former groker as well as his experience teaching at Codeup.</p>
<p>Show Notes:</p>
<ul>
<li><a href="http://www.codeup.com/">Codeup</a></li>
</ul>
</description>
</item>
<item>
<title>06 - Professional Development</title>
<link>http://www.grok-interactive.com/podcast/professional-development</link>
<pubDate>Tue, 13 Oct 2015 00:00:00 -0400</pubDate>
<author> ()</author>
<guid>http://www.grok-interactive.com/podcast/professional-development</guid>
<description><p>Continuing education is especially important in this industry. Our clients look to us as the experts that they hire to solve problems. It&#39;s our job to not only solve those problems but to stay up to date and learn new technologies and frameworks in order to provide the best most current solution.</p>
<p>Failing to stay current or up to date can cause you as a developer to stagnate and/or get stuck in your career doing maintenance forever. Discovering your best and most effective method to learn is a must in order to stay up to date.</p>
<p>Show Notes:</p>
<ul>
<li><a href="https://www.youtube.com/user/Confreaks">ConFreaks</a></li>
<li><a href="https://www.youtube.com/watch?v=DC-pQPq0acs">Refactoring from Good to Great by Ben Orenstein</a></li>
<li><a href="https://www.youtube.com/watch?v=OMPfEXIlTVE">Nothing is Something by Sandi Metz</a></li>
</ul>
</description>
</item>
<item>
<title>Functional enumerators in Ruby</title>
<link>http://www.grok-interactive.com/blog/functional-enumerators-in-ruby/</link>
<pubDate>Mon, 12 Oct 2015 00:00:00 -0400</pubDate>
<author>christopher.moeller@grok-interactive.com (Christopher Moeller)</author>
<guid>http://www.grok-interactive.com/blog/functional-enumerators-in-ruby</guid>
<description><h3>Functional enumerators in Ruby</h3>
<p>Ruby is a beautifully designed object-oriented programming language with
heritage in Smalltalk, Perl, and Lisp among others. People are attracted to many
different features in Ruby: almost everything in Ruby is an object, we have
terse constructs for iterating through collections, and it&#39;s friendly to
beginners and experts alike. One of the reasons people love programming in Ruby
is the <code>Enumerable</code> module. While developers can loop though collections using
more traditional constructs like <code>for</code> and <code>while</code> loops, idiomatic Ruby
typically uses the methods found in <code>Enumerable</code>.</p>
<p><code>Enumerable</code> adds a lot of methods to the class where it&#39;s included, and all of
these methods are based on the <code>each</code> method. I&#39;ve read (and also written) a lot
of iterative code using <code>each</code> when there&#39;s a more elegant, explicit, and
performant method that I could have used. Using the <code>Array</code> and <code>Range</code> classes
as examples, I&#39;ll start with a brief overview of <code>each</code> and <code>map</code> and conclude
with an overview of <code>reduce</code>.</p>
<h4>What makes some of the iterators functional?</h4>
<p>One of the many principles of functional programming is higher order functions.
For a function to be a &quot;higher order&quot; function, it needs to either receive a
function as an argument or return a function as the return value. The main idea
is using functions as values.</p>
<p>In JavaScript we call these &quot;anonymous functions&quot; while in Ruby we call them
lambdas and procs. In most functional languages I&#39;ve surveyed and explored,
they&#39;re just called lambdas. It&#39;s outside the scope of this post to go into
detail on lambdas and procs in Ruby but there are a <em>ton</em> of great resources
online that explain what they are and how to use them.</p>
<p>Now that we understand why some of Enumerable&#39;s iterators are functional, let&#39;s
look at how to use them in our code.</p>
<h4>Iterating with each</h4>
<p>As I mentioned, Ruby allows developers to iterate through collections with
traditional, imperative-style constructs like <code>while</code> and <code>for</code>. For example,
using a <code>Range</code> with <code>for</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="k">for</span> <span class="n">n</span> <span class="k">in</span> <span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="nb">puts</span> <span class="n">n</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="k">end</span>
<span class="mi">1</span>
<span class="mi">2</span>
<span class="mi">3</span>
<span class="mi">4</span>
<span class="mi">5</span>
<span class="mi">6</span>
<span class="o">=&gt;</span> <span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span>
</code></pre></div>
<p>We have a range of integers from 0 to 5 and we print each number plus 1 to the
console. This works just fine. This is how we would get the same result using
<code>each</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">1</span><span class="o">*</span> <span class="nb">puts</span> <span class="n">n</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="k">end</span>
<span class="mi">1</span>
<span class="mi">2</span>
<span class="mi">3</span>
<span class="mi">4</span>
<span class="mi">5</span>
<span class="mi">6</span>
<span class="o">=&gt;</span> <span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span>
</code></pre></div>
<p>This does the exact same thing as the code with <code>for</code>: we print the values of
every element in the collection incremented by 1. So why do many Rubyists say
using <code>each</code> is better than <code>for</code>? Personally, I think it has to do with the
<em>mindset</em> of the developer. Using <code>each</code> makes me think more about <em>what</em> I want
the computer to do and less about <em>how</em> I want it to perform the operation.
Thinking about the result I want creates a layer of abstraction so I can focus
on the bigger picture of my current task.</p>
<p>This additional layer of abstraction can be shown by using another one of
Enumerable&#39;s methods: <code>map</code>.</p>
<h4>Using Enumerable&#39;s map</h4>
<p>We can print each value of an array with <code>each</code>, but what if we wanted to save
these incremented values into a new array? We know we should use <code>each</code> to
iterate of the collection instead of <code>for</code>, so let&#39;s take a look at one approach
to saving the incremented values using <code>each</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">incremented</span> <span class="o">=</span> <span class="o">[]</span>
<span class="o">=&gt;</span> <span class="o">[]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">1</span><span class="o">*</span> <span class="n">incremented</span> <span class="o">&lt;&lt;</span> <span class="n">n</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">004</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="k">end</span>
<span class="o">=&gt;</span> <span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">005</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">incremented</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="o">]</span>
</code></pre></div>
<p>This works. We start with a variable to hold our new collection (<code>incremented</code>),
iterate through each number in the range, and append each incremented number to
the new array. However, in writing this example I found myself caught in the
details of performing each step of the task instead of simply requesting the
result I wanted: an array of incremented integers. From the world of functional
programming, <code>Enumerable</code> provides <code>map</code> (also aliased as <code>collect</code>) to better
manage this:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">incremented</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">1</span><span class="o">*</span> <span class="n">n</span> <span class="o">+</span> <span class="mi">1</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="k">end</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="o">]</span>
</code></pre></div>
<p>The result of calling <code>map</code> on the collection is exactly what we&#39;re looking for.
In this case we save it to the <code>incremented</code> variable. Also, as Enumerable is
tuned for better performance, we get those benefits for free. I don&#39;t care <em>how</em>
the computer creates this new <code>Array</code>, I just care that each value is
incremented by <code>1</code>.</p>
<p>Using <code>map</code> also has a more succinct way of expressing this operation using
<code>Symbol#to_proc</code> and Fixnum&#39;s <code>succ</code> method (you can also use <code>next</code> if you
prefer):</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">incremented</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:succ</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="o">]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="sx">%w(foo bar baz)</span><span class="o">.</span><span class="n">map</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:capitalize</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="s2">&quot;Foo&quot;</span><span class="p">,</span> <span class="s2">&quot;Bar&quot;</span><span class="p">,</span> <span class="s2">&quot;Baz&quot;</span><span class="o">]</span>
</code></pre></div>
<p>In case you haven&#39;t seen it yet, you can use the unary <code>&amp;</code> operator with a
symbol as the method name to call on each member of the collection. I won&#39;t go
into the details here but you can
<a href="http://ruby-doc.org/core-2.2.3/Symbol.html#method-i-to_proc">check out the <code>Symbol#to_proc</code> documentation</a>.
(Keep in mind this only works if every element in the collection responds to the
method. Otherwise you&#39;ll get a <code>NoMethodError</code>.)</p>
<p>This is pretty great! We can take a collection and easily create another
collection with some modified values. In the same vein of <code>map</code>, Enumerable
provides <code>find</code>, <code>select</code>, <code>reject</code>, etc. All are ways of building new
collections from an existing collection and applying the same method to each of
the collection&#39;s elements.</p>
<p>Sometimes we don&#39;t want a new collection, though. Instead, we want a singular
value back. Ruby again borrows from the functional programming world again to
provide <code>reduce</code> (aliased as <code>inject</code>): we take a collection and <em>reduce</em> all of
it&#39;s values into a single value. Let&#39;s take a look at it.</p>
<h4>Using reduce</h4>
<p>I&#39;ll start with an example:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">reduce</span> <span class="p">{</span> <span class="o">|</span><span class="n">accumulator</span><span class="p">,</span> <span class="n">n</span><span class="o">|</span> <span class="n">accumulator</span> <span class="o">+</span> <span class="n">n</span> <span class="p">}</span>
<span class="o">=&gt;</span> <span class="mi">15</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="mi">0</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">accumulator</span><span class="p">,</span> <span class="n">n</span><span class="o">|</span> <span class="n">accumulator</span> <span class="o">+</span> <span class="n">n</span> <span class="p">}</span>
<span class="o">=&gt;</span> <span class="mi">15</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:+</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="mi">15</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">004</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="o">&amp;</span><span class="ss">:+</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="mi">20</span>
</code></pre></div>
<p>Here we take the same range we&#39;ve been using <code>0..5</code> and add all of its numbers
together to produce a single integer: <code>15</code>. Let&#39;s look into the first example
with the block a bit more: <code>(0..5).reduce { |accumulator, n| accumulator + n }</code>.</p>
<p>In this form, <code>reduce</code> doesn&#39;t take any arguments except the block. The block
must declare an accumulator variable (I usually call it <code>acc</code> but opted for the
more descriptive <code>accumulator</code> name in this case) and a variable for the member
of the collection for each iteration, <code>n</code> in this case. <code>reduce</code> then sets the
value of <code>accumulator</code> to the result of the block, in this case adding the
current value of <code>accumulator</code> to <code>n</code>. This happens until all the values in the
collection have been reached.</p>
<p>In the second form of the method (when <code>0</code> is passed to <code>reduce</code>), <code>0</code> is the
initial value for <code>accumulator</code>, which is why the result is the same as the
example above it.</p>
<p>The last two forms use the more terse (and I&#39;d argue more readable)
<code>Symbol#to_proc</code> syntax, one without explicitly setting the accumulators initial
value and one with setting the initial value to <code>5</code>.</p>
<p>Here&#39;s another way to get the same result using <code>each</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">accumulator</span> <span class="o">=</span> <span class="mi">0</span>
<span class="o">=&gt;</span> <span class="mi">0</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">result</span> <span class="o">=</span> <span class="p">(</span><span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span><span class="p">)</span><span class="o">.</span><span class="n">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">003</span><span class="p">:</span><span class="mi">1</span><span class="o">*</span> <span class="n">accumulator</span> <span class="o">=</span> <span class="n">accumulator</span> <span class="o">+</span> <span class="n">n</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">004</span><span class="p">:</span><span class="mi">1</span><span class="o">&gt;</span> <span class="k">end</span>
<span class="o">=&gt;</span> <span class="mi">0</span><span class="o">.</span><span class="n">.</span><span class="mi">5</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">005</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">accumulator</span>
<span class="o">=&gt;</span> <span class="mi">15</span>
</code></pre></div>
<p>As with our example for using <code>map</code> instead of <code>each</code> (when appropriate), using
<code>reduce</code> frees us from telling the computer how to do the calculation. We tell
it what we want in the end: the sum of every integer in the collection.</p>
<h4>One more thing: Lazy Enumeration</h4>
<p>Using a dynamically typed language like Ruby, we can create arrays with values
of mixed types. We also encourage
<a href="https://en.wikipedia.org/wiki/Duck_typing">duck typing</a> over checking an
object&#39;s type. However, this language feature and programming best practice can
create some unexpected consequences. For example, the <code>+</code> method is defined for
both Strings and Integers but they can&#39;t be mixed. If we have an array of both
Integers and Strings and call <code>reduce</code> on the array with <code>+</code>, we get a
<code>TypeError</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span> <span class="o">=</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="ss">TypeError</span><span class="p">:</span> <span class="nb">String</span> <span class="n">can</span><span class="s1">&#39;t be coerced into Fixnum</span>
<span class="s1"> from (irb):6:in `+&#39;</span>
<span class="n">from</span> <span class="p">(</span><span class="n">irb</span><span class="p">):</span><span class="mi">6</span><span class="ss">:in</span> <span class="sb">`each&#39;</span>
<span class="sb"> from (irb):6:in `</span><span class="n">reduce</span><span class="s1">&#39;</span>
<span class="s1"> from (irb):6</span>
<span class="s1"> from /Users/cmoel/.rubies/ruby-2.2.3/bin/irb:11:in `&lt;main&gt;&#39;</span>
</code></pre></div>
<p>This happens because <code>+</code> is defined for both Integers and Strings but it has a
different (and incompatible) meaning for each class: addition for <code>Integer</code> and
concatenation for <code>String</code>. While the solution to this problem is completely
dependent on the needs of the application, we&#39;ll say we do not need the <code>String</code>
values and will only keep the <code>Integer</code> values. We can do this with <code>select</code>:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span> <span class="o">=</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span><span class="o">.</span><span class="n">select</span><span class="p">{</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span> <span class="n">n</span><span class="o">.</span><span class="n">is_a?</span><span class="p">(</span><span class="nb">Integer</span><span class="p">)</span> <span class="p">}</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
</code></pre></div>
<p>Now we can do the same <code>reduce</code> we did previously:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span> <span class="o">=</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span><span class="o">.</span><span class="n">select</span><span class="p">{</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span> <span class="n">n</span><span class="o">.</span><span class="n">is_a?</span><span class="p">(</span><span class="nb">Integer</span><span class="p">)</span> <span class="p">}</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:+</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="mi">15</span>
</code></pre></div>
<p>There&#39;s a problem here, though. Each time we use one of the Enumerable
functions, we fully traverse the collection. In the last example, we run through
the collection twice, once for <code>select</code> and once for <code>reduce</code>. If <code>mixed</code> was a
very large collection, we would eventually run out of memory and the program
would crash. To solve this, in Ruby 2.0, <code>Enumerator::Lazy</code> was introduced. I
won&#39;t give a full explanation in this post, but the gist is to call <code>lazy</code> on
the collection before using any <code>Enumerable</code> methods:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span> <span class="o">=</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span><span class="o">.</span><span class="n">lazy</span><span class="o">.</span><span class="n">select</span><span class="p">{</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span> <span class="n">n</span><span class="o">.</span><span class="n">is_a?</span><span class="p">(</span><span class="nb">Integer</span><span class="p">)</span> <span class="p">}</span><span class="o">.</span><span class="n">reduce</span><span class="p">(</span><span class="o">&amp;</span><span class="ss">:+</span><span class="p">)</span>
<span class="o">=&gt;</span> <span class="mi">120</span>
</code></pre></div>
<p>Removing the call to <code>reduce</code>, we can see what we&#39;re actually building:</p>
<div class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="err">$</span> <span class="n">irb</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">001</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span> <span class="o">=</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="o">=&gt;</span> <span class="o">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="s2">&quot;foo&quot;</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="s2">&quot;bar&quot;</span><span class="p">,</span> <span class="mi">5</span><span class="o">]</span>
<span class="n">irb</span><span class="p">(</span><span class="n">main</span><span class="p">):</span><span class="mo">002</span><span class="p">:</span><span class="mi">0</span><span class="o">&gt;</span> <span class="n">mixed</span><span class="o">.</span><span class="n">lazy</span><span class="o">.</span><span class="n">select</span><span class="p">{</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span> <span class="n">n</span><span class="o">.</span><span class="n">is_a?</span><span class="p">(</span><span class="nb">Integer</span><span class="p">)</span> <span class="p">}</span>
<span class="o">=&gt;</span> <span class="c1">#&lt;Enumerator::Lazy: #&lt;Enumerator::Lazy: [1, 2, &quot;foo&quot;, 3, 4, &quot;bar&quot;, 5]&gt;:select&gt;</span>
</code></pre></div>
<p>Calling <code>reduce</code> forces our <code>Enumerator::Lazy</code> to evaluate and perform the
reduction. If we simply wanted a new array, we could call <code>to_a</code> on the
<code>Enumerator::Lazy</code> to force evaluation.
<a href="http://ruby-doc.org/core-2.2.3/Enumerator/Lazy.html">Check out the docs</a> for
more details. Pat Shaughnessy has also written a very
<a href="http://patshaughnessy.net/2013/4/3/ruby-2-0-works-hard-so-you-can-be-lazy">detailed blog post on <code>Enumerator::Lazy</code></a>
that I wholeheartedly recommend.</p>
<h4>Conclusion</h4>
<p>Ruby allows developers to write clear concise code following object-oriented
principles and using procedural or functional principles as well. To me, using
the more functionally-inspired methods from <code>Enumerable</code> makes the developer&#39;s
intent more clear without getting stuck in the details. It also allows
developers to think at a higher level about <em>what</em> they want instead of telling
the computer <em>how</em> to create what they want, making for more declarative code.</p>
<p>What do you think? Are there any <code>Enumerable</code> methods you tend to reach for over
others? Which ones are your favorites?</p>
</description>
</item>
<item>
<title>05 - Pleasure Reading for Geeks</title>
<link>http://www.grok-interactive.com/podcast/pleasure-reading-for-geeks</link>
<pubDate>Tue, 06 Oct 2015 00:00:00 -0400</pubDate>
<author> ()</author>
<guid>http://www.grok-interactive.com/podcast/pleasure-reading-for-geeks</guid>
<description><p>Spending many hours in front of a screen each day makes many developers look to
books as one of their preferred ways of entertainment. While many of us like to
stay in the realm of science fiction, many geeks love fantasy. We can step away
from the strict rules we have with programming and explore other worlds with
magic and imaginative creatures. Or sometimes we step back in history to better
understand who we are and where we came from.</p>
<p>Whatever your preferred genres are, there are some must-read classics as well as
newer works that are great reads. Listen to a few of the Grokers talk about
their favorite genres. You&#39;ll also get some great book recommendations to add to
your reading list.</p>
</description>
</item>
<item>
<title>Importing Large CSV files with PHP Part 3: Processing and Validating data in background</title>
<link>http://www.grok-interactive.com/blog/import-large-csv-into-mysql-with-php-part-3-background-import/</link>
<pubDate>Tue, 06 Oct 2015 00:00:00 -0400</pubDate>
<author>anton.domratchev@grok-interactive.com (Anton Domratchev)</author>
<guid>http://www.grok-interactive.com/blog/import-large-csv-into-mysql-with-php-part-3-background-import</guid>
<description><p>So far we&#39;ve learned how we can import a large CSV file using one MySQL query. We also learned how we can validate the structure of the CSV file prior to import. In some cases using these two methods might work for imports where the structure of the CSV data matches the data of your application. However, in other cases you may need to pre process the data in order to fit it into the data structure of your application. You will need to implement background processing and validation to handle the data without delaying the HTTP request. In this blog post I will describe how to create an artisan command wich will be responsible for processing and validating data from an imported CSV, and a simple guide to seting up a cron job. </p>
<p>The basic idea in this implementation is that we will quickly import the entire CSV file into a temprorary table keep the file&#39;s original structure line by line. Next we will set up a cron job to run periodically and check this table for new records to be processed. We will also keep a simple log of each import attempt in order to get feedback on the data being processed. For this example our CSV will contain a list of users as well as a list of their favorite movies and music.</p>
<p>First we need to modify the CsvImporter to record the data into a temporary table. We will set up two modelsone for keeping track of each file and another for keeping each row of the file</p>
<hr>
<h3>Models</h3>
<p>This model will keep record of each CSV file we upload</p>
<div class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp">&lt;?php</span>
<span class="k">class</span> <span class="nc">CsvImport</span> <span class="k">extends</span> <span class="nx">Eloquent</span>
<span class="p">{</span>
<span class="k">protected</span> <span class="nv">$softDelete</span> <span class="o">=</span> <span class="k">false</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$fillable</span> <span class="o">=</span> <span class="p">[</span>
<span class="s1">&#39;original_filename&#39;</span><span class="p">,</span>
<span class="s1">&#39;status&#39;</span><span class="p">,</span>
<span class="s1">&#39;row_count&#39;</span>
<span class="p">];</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">csvRow</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">hasMany</span><span class="p">(</span><span class="s1">&#39;csvRows&#39;</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="cp">?&gt;</span><span class="x"></span>
</code></pre></div>
<p>Now each time we upload a CSV file a <code>CsvImport</code> record will be created to reference that file, it will keep the status of the import as well as file&#39;s original file name this way we can keep track of the CSV file import. </p>
<div class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp">&lt;?php</span>
<span class="k">class</span> <span class="nc">CsvRow</span> <span class="k">extends</span> <span class="nx">Eloquent</span>
<span class="p">{</span>
<span class="k">protected</span> <span class="nv">$softDelete</span> <span class="o">=</span> <span class="k">false</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$fillable</span> <span class="o">=</span> <span class="p">[</span>
<span class="s1">&#39;csv_import_id&#39;</span><span class="p">,</span>
<span class="s1">&#39;header&#39;</span><span class="p">,</span>
<span class="s1">&#39;content&#39;</span>
<span class="p">];</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">csvImport</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">belongsTo</span><span class="p">(</span><span class="s1">&#39;csvImport&#39;</span><span class="p">)</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="cp">?&gt;</span><span class="x"></span>
</code></pre></div>
<p>This model will keep record of each row from the file including the header. For each row in the CSV we will create a <code>CsvRow</code> record containing the id of the <code>CsvImport</code> as well as header and row content. This way we will have a point of reference for each row of data.</p>
<hr>
<h3>Controller</h3>
<p>Now we need to modify our controller to match our new import structure. We will also inject the validator as dependency on the controller. (I&#39;ve removed inline comments to keep the code samples shorter)</p>
<div class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp">&lt;?php</span>
<span class="k">namespace</span> <span class="nx">Acme\Controllers</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">App</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">Acme\Importing\CsvImporter</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">Acme\Importing\CsvImportValidator</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">CsvImport</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">Input</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">CsvImportController</span> <span class="k">extends</span> <span class="nx">BaseController</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$csv_import_validator</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">__construct</span><span class="p">(</span><span class="nx">CsvImportValidator</span> <span class="nv">$csv_import_validator</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import_validator</span> <span class="o">=</span> <span class="nv">$csv_import_validator</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">store</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">Input</span><span class="o">::</span><span class="na">hasFile</span><span class="p">(</span><span class="s1">&#39;csv_import&#39;</span><span class="p">))</span> <span class="p">{</span>
<span class="nv">$csv_file</span> <span class="o">=</span> <span class="nx">Input</span><span class="o">::</span><span class="na">file</span><span class="p">(</span><span class="s1">&#39;csv_import&#39;</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$csv_file</span><span class="o">-&gt;</span><span class="na">isValid</span><span class="p">())</span> <span class="p">{</span>
<span class="nv">$validator</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import_validator</span><span class="o">-&gt;</span><span class="na">validate</span><span class="p">(</span><span class="nv">$csv_file</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$validator</span><span class="o">-&gt;</span><span class="na">fails</span><span class="p">())</span> <span class="p">{</span>
<span class="k">return</span> <span class="nx">Redirect</span><span class="o">::</span><span class="na">back</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">withErrors</span><span class="p">(</span><span class="nv">$validator</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$original_filename</span> <span class="o">=</span> <span class="nv">$csv_file</span><span class="o">-&gt;</span><span class="na">getClientOriginalName</span><span class="p">();</span>
<span class="nv">$csv_import</span> <span class="o">=</span> <span class="nx">CsvImport</span><span class="o">::</span><span class="na">create</span><span class="p">([</span>
<span class="s1">&#39;original_filename&#39;</span> <span class="o">=&gt;</span> <span class="nv">$original_filename</span><span class="p">,</span>
<span class="s1">&#39;status&#39;</span> <span class="o">=&gt;</span> <span class="s1">&#39;pending&#39;</span><span class="p">,</span>
<span class="s1">&#39;row_count&#39;</span> <span class="o">=&gt;</span> <span class="mi">0</span>
<span class="p">]);</span>
<span class="nv">$csv_importer</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">CsvFileImporter</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$csv_import</span><span class="o">-&gt;</span><span class="na">row_count</span> <span class="o">=</span> <span class="nv">$csv_importer</span><span class="o">-&gt;</span><span class="na">import</span><span class="p">(</span><span class="nv">$csv_file</span><span class="p">,</span> <span class="nv">$csv_import</span><span class="o">-&gt;</span><span class="na">id</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$message</span> <span class="o">=</span> <span class="s2">&quot;</span><span class="si">{</span><span class="nv">$csv_import</span><span class="o">-&gt;</span><span class="na">row_count</span><span class="si">}</span><span class="s2"> rows were imported!&quot;</span><span class="p">;</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nv">$message</span> <span class="o">=</span> <span class="s1">&#39;Your file did not import&#39;</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nv">$message</span> <span class="o">=</span> <span class="s1">&#39;You must provide a CSV file for import.&#39;</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">Redirect</span><span class="o">::</span><span class="na">back</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">with</span><span class="p">(</span><span class="s1">&#39;message&#39;</span><span class="p">,</span> <span class="nv">$message</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="cp">?&gt;</span><span class="x"></span>
</code></pre></div>
<p>Next we will need to update our CsvFileImporter to import the data inside of the CSV file into our temporary table and connect it to a single CsvImport record. This will help us keep track of what we need to process in our command. Once again I&#39;ve removed the inline comments for compactness. </p>
<hr>
<h3>CSV Importer</h3>
<div class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp">&lt;?php</span>
<span class="k">namespace</span> <span class="nx">Acme\Importing</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">DB</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">CsvFileImporter</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">import</span><span class="p">(</span><span class="nv">$csv_import</span><span class="p">,</span> <span class="nv">$csv_import_id</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$moved_file</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">moveFile</span><span class="p">(</span><span class="nv">$csv_import</span><span class="p">);</span>
<span class="nv">$normalized_file</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">normalize</span><span class="p">(</span><span class="nv">$moved_file</span><span class="p">);</span>
<span class="nv">$csv_header</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">getCSVHeader</span><span class="p">(</span><span class="nv">$moved_file</span><span class="p">);</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">importFileContents</span><span class="p">(</span><span class="nv">$normalized_file</span><span class="p">,</span> <span class="nv">$csv_import_id</span><span class="p">,</span> <span class="nv">$csv_header</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">function</span> <span class="nf">moveFile</span><span class="p">(</span><span class="nv">$csv_import</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">is_dir</span><span class="p">(</span><span class="nv">$destination_directory</span> <span class="o">=</span> <span class="nx">storage_path</span><span class="p">(</span><span class="s1">&#39;imports/tmp&#39;</span><span class="p">))</span> <span class="p">{</span>
<span class="nb">chmod</span><span class="p">(</span><span class="nv">$destination_directory</span><span class="p">,</span> <span class="mo">0755</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nb">mkdir</span><span class="p">(</span><span class="nv">$destination_directory</span><span class="p">,</span> <span class="mo">0755</span><span class="p">,</span> <span class="k">true</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$original_file_name</span> <span class="o">=</span> <span class="nv">$csv_import</span><span class="o">-&gt;</span><span class="na">getClientOriginalName</span><span class="p">();</span>
<span class="k">return</span> <span class="nv">$csv_import</span><span class="o">-&gt;</span><span class="na">move</span><span class="p">(</span><span class="nv">$destination_directory</span><span class="p">,</span> <span class="nv">$original_file_name</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">function</span> <span class="nf">normalize</span><span class="p">(</span><span class="nv">$file_path</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$string</span> <span class="o">=</span> <span class="o">@</span><span class="nb">file_get_contents</span><span class="p">(</span><span class="nv">$file_path</span><span class="p">);</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nv">$string</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nv">$file_path</span><span class="p">;</span>
<span class="p">}</span>
<span class="nv">$string</span> <span class="o">=</span> <span class="nb">preg_replace</span><span class="p">(</span><span class="s1">&#39;~\r\n?~&#39;</span><span class="p">,</span> <span class="s2">&quot;</span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="nv">$string</span><span class="p">);</span>
<span class="nb">file_put_contents</span><span class="p">(</span><span class="nv">$file_path</span><span class="p">,</span> <span class="nv">$string</span><span class="p">);</span>
<span class="k">return</span> <span class="nv">$file_path</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">function</span> <span class="nf">importFileContents</span><span class="p">(</span><span class="nv">$file_path</span><span class="p">,</span> <span class="nv">$csv_import_id</span><span class="p">,</span> <span class="nv">$csv_header</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$query</span> <span class="o">=</span> <span class="nb">sprintf</span><span class="p">(</span><span class="s2">&quot;LOAD DATA LOCAL INFILE &#39;%s&#39; INTO TABLE csv_rows </span>
<span class="s2"> LINES TERMINATED BY &#39;</span><span class="se">\\</span><span class="s2">n&#39;</span>
<span class="s2"> FIELDS TERMINATED BY &#39;</span><span class="se">\\</span><span class="s2">n&#39; </span>
<span class="s2"> IGNORE 1 LINES (`content`) </span>
<span class="s2"> SET csv_import_id = &#39;%s&#39;, header = &#39;%s&#39;&quot;</span><span class="p">,</span> <span class="nb">addslashes</span><span class="p">(</span><span class="nv">$file_path</span><span class="p">),</span> <span class="nv">$csv_import_id</span><span class="p">,</span> <span class="nv">$csv_header</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">DB</span><span class="o">::</span><span class="na">connection</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">getpdo</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">exec</span><span class="p">(</span><span class="nv">$query</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">private</span> <span class="k">function</span> <span class="nf">getCSVHeader</span><span class="p">(</span><span class="nv">$file</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">((</span><span class="nv">$file</span> <span class="o">=</span> <span class="nb">fopen</span><span class="p">(</span><span class="nv">$file</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">))</span> <span class="o">===</span> <span class="k">false</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nx">\Exception</span><span class="p">(</span><span class="s2">&quot;The file (</span><span class="si">{</span><span class="nv">$file</span><span class="si">}</span><span class="s2">) could not be opened for reading&quot;</span><span class="p">);</span>
<span class="p">}</span>
<span class="nb">ini_set</span><span class="p">(</span><span class="s1">&#39;auto_detect_line_endings&#39;</span><span class="p">,</span> <span class="k">true</span><span class="p">);</span>
<span class="nv">$header</span> <span class="o">=</span> <span class="nb">fgets</span><span class="p">(</span><span class="nv">$file</span><span class="p">);</span>
<span class="nb">fclose</span><span class="p">(</span><span class="nv">$file</span><span class="p">);</span>
<span class="k">return</span> <span class="nv">$header</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="cp">?&gt;</span><span class="x"></span>
</code></pre></div>
<p>We&#39;ve added and paramter to the import method, this will let us attach all of our content records to a <code>CsvImport</code> this way we will be ablet to track the status of the import and in the future keep track of errors, duplicates, and validation erros. Additionally, we&#39;ve added <code>getCSVHeader</code> method which will extract the header from the csv, this will allow us to save the header with each row of the data. We&#39;ve also modified the query to save the entire row as a field as well as csv_import_id and the header.</p>
<hr>
<h3>Artisan Command</h3>
<p>Now lets work on our artisan command. As I described earlier, the artisan command will help us do our record processing and validation through a command line. We will start by creating our artisan command</p>
<div class="highlight"><pre><code class="language-php" data-lang="php"><span class="cp">&lt;?php</span>
<span class="k">namespace</span> <span class="nx">Acme\ArtisanCommands</span><span class="p">;</span>
<span class="k">use</span> <span class="nx">Illuminate\Console\Command</span><span class="p">;</span>
<span class="k">class</span> <span class="nc">ProcessImportedCsvCommand</span> <span class="k">extends</span> <span class="nx">Command</span>
<span class="p">{</span>
<span class="sd">/**</span>
<span class="sd"> * The console command name.</span>
<span class="sd"> *</span>
<span class="sd"> * @var string</span>
<span class="sd"> */</span>
<span class="k">private</span> <span class="nv">$name</span> <span class="o">=</span> <span class="s1">&#39;acme:ProcessImportedCsvCommand&#39;</span><span class="p">;</span>
<span class="sd">/**</span>
<span class="sd"> * The console command description.</span>
<span class="sd"> *</span>
<span class="sd"> * @var string</span>
<span class="sd"> */</span>
<span class="k">private</span> <span class="nv">$description</span> <span class="o">=</span> <span class="s1">&#39;Command line file procesing command to process imported CSV file into the database&#39;</span><span class="p">;</span>
<span class="sd">/**</span>
<span class="sd"> * CSV import collection object</span>
<span class="sd"> *</span>
<span class="sd"> * @var CsvImport</span>
<span class="sd"> */</span>
<span class="k">private</span> <span class="nv">$csv_import</span><span class="p">;</span>
<span class="sd">/**</span>
<span class="sd"> * Execute the console command.</span>
<span class="sd"> *</span>
<span class="sd"> * @return mixed</span>
<span class="sd"> */</span>
<span class="k">public</span> <span class="k">function</span> <span class="nf">fire</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">processFileImport</span><span class="p">();</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">cleanUp</span><span class="p">();</span>
<span class="p">}</span>
<span class="sd">/**</span>
<span class="sd"> * Process file import data into application data</span>
<span class="sd"> *</span>
<span class="sd"> * @return void</span>
<span class="sd"> */</span>
<span class="k">private</span> <span class="k">function</span> <span class="nf">processFileImport</span><span class="p">()</span>
<span class="p">{</span>
<span class="c1">// Disable query log to conserve on memory</span>
<span class="nx">DB</span><span class="o">::</span><span class="na">connection</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">disableQueryLog</span><span class="p">();</span>
<span class="c1">// Find first pending file import and eager load the csvRows</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span> <span class="o">=</span> <span class="nx">CsvImport</span><span class="o">::</span><span class="na">where</span><span class="p">(</span><span class="s1">&#39;status&#39;</span><span class="p">,</span> <span class="s1">&#39;pending&#39;</span><span class="p">)</span>
<span class="o">-&gt;</span><span class="na">with</span><span class="p">(</span><span class="s1">&#39;csvRows&#39;</span><span class="p">)</span>
<span class="o">-&gt;</span><span class="na">first</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="p">)</span> <span class="p">{</span>
<span class="nb">printf</span><span class="p">(</span><span class="s2">&quot;%s </span><span class="se">\n</span><span class="s2">&quot;</span><span class="p">,</span> <span class="s2">&quot;Nothing to process.&quot;</span><span class="p">);</span>
<span class="k">exit</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// Change the status of the import to avoid it being processed again.</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="o">-&gt;</span><span class="na">status</span> <span class="o">=</span> <span class="s1">&#39;processing&#39;</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="o">-&gt;</span><span class="na">save</span><span class="p">();</span>
<span class="k">if</span> <span class="p">(</span><span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="o">-&gt;</span><span class="na">fileImportContent</span><span class="p">))</span> <span class="p">{</span>
<span class="c1">// Get the header from the import</span>
<span class="nv">$header</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="o">-&gt;</span><span class="na">fileImportContent</span><span class="o">-&gt;</span><span class="na">first</span><span class="p">()</span><span class="o">-&gt;</span><span class="na">header</span><span class="p">;</span>
<span class="c1">// Flip the values to keys and explode into array</span>
<span class="nv">$header</span> <span class="o">=</span> <span class="nb">array_flip</span><span class="p">(</span><span class="nb">explode</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">,</span> <span class="nv">$header</span><span class="p">));</span>
<span class="k">foreach</span> <span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="na">csv_import</span><span class="o">-&gt;</span><span class="na">fileImportContent</span> <span class="k">as</span> <span class="nv">$row</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// Convert the raw row data into array</span>
<span class="nv">$row_array</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">,</span> <span class="nv">$row</span><span class="o">-&gt;</span><span class="na">content</span><span class="p">);</span>
<span class="c1">// Build array of import properties</span>
<span class="nv">$import_array</span> <span class="o">=</span> <span class="nv">$header</span><span class="p">;</span>
<span class="c1">// Match data array to the header array</span>
<span class="k">foreach</span> <span class="p">(</span><span class="nv">$import_array</span> <span class="k">as</span> <span class="nv">$index</span> <span class="o">=&gt;</span> <span class="nv">$key</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$import_array</span><span class="p">[</span><span class="nv">$index</span><span class="p">]</span> <span class="o">=</span> <span class="nv">$row_array</span><span class="p">[</span><span class="nv">$key</span><span class="p">];</span>
<span class="p">}</span>
<span class="sd">/** </span>
<span class="sd"> * Here we can perform additional validation</span>
<span class="sd"> * for each data field and save it to the database</span>
<span class="sd"> */</span>
<span class="c1">// Build array of parameters for our user import</span>
<span class="nv">$user</span> <span class="o">=</span> <span class="nx">User</span><span class="o">::</span><span class="na">firstOrCreate</span><span class="p">([</span>
<span class="s1">&#39;first_name&#39;</span> <span class="o">=&gt;</span> <span class="nv">$import_array</span><span class="p">[</span><span class="s1">&#39;first_name&#39;</span><span class="p">],</span>
<span class="s1">&#39;last_name&#39;</span> <span class="o">=&gt;</span> <span class="nv">$import_array</span><span class="p">[</span><span class="s1">&#39;last_name&#39;</span><span class="p">],</span>
<span class="s1">&#39;email&#39;</span> <span class="o">=&gt;</span> <span class="nv">$import_array</span><span class="p">[</span><span class="s1">&#39;email&#39;</span><span class="p">]</span>
<span class="p">]);</span>
<span class="nv">$movies</span> <span class="o">=</span> <span class="nx">Movies</span><span class="o">::</span><span class="na">firstOrCreate</span><span class="p">([</span>
<span class="s1">&#39;user_id&#39;</span> <span class="o">=&gt;</span> <span class="nv">$user</span><span class="o">-&gt;</span><span class="na">id</span><span class="p">,</span>
<span class="s1">&#39;movie_list&#39;</span> <span class="o">=&gt;</span> <span class="nv">$import_array</span><span class="p">[</span><span class="s1">&#39;movies&#39;</span><span class="p">]</span>
<span class="p">]);</span>
<span class="nv">$music</span> <span class="o">=</span> <span class="nx">Music</span><span class="o">::</span><span class="na">firstOrCreate</span><span class="p">([</span>
<span class="s1">&#39;user_id&#39;</span> <span class="o">=&gt;</span> <span class="nv">$user</span><span class="o">-&gt;</span><span class="na">id</span><span class="p">,</span>