Stuff I like from week 2 of Google Plus meme- animated GIFS,jokes,nice photos are just some of them-
Here is week 1 in case you missed it
https://decisionstats.com/best-of-google-plus-week-1-top10/
Tag: Random
The Top Statisticians in the World
http://en.wikipedia.org/wiki/John_Tukey
John Tukey
From Wikipedia, the free encyclopedia
| John Tukey | |
|---|---|
John Wilder Tukey
|
|
| Born | June 16, 1915 New Bedford, Massachusetts, USA |
| Died | July 26, 2000 (aged 85) New Brunswick, New Jersey |
| Residence | United States |
| Nationality | American |
| Fields | Mathematician |
| Institutions | Bell Labs Princeton University |
| Alma mater | Brown University Princeton University |
| Doctoral advisor | Solomon Lefschetz |
| Doctoral students | Frederick Mosteller Kai Lai Chung |
| Known for | FFT algorithm Box plot Coining the term ‘bit’ |
| Notable awards | Samuel S. Wilks Award (1965) National Medal of Science (USA) in Mathematical, Statistical, and Computational Sciences (1973) Shewhart Medal (1976) IEEE Medal of Honor (1982) Deming Medal (1982) James Madison Medal (1984) Foreign Member of the Royal Society(1991) |
John Wilder Tukey ForMemRS[1] (June 16, 1915 – July 26, 2000) was an American statistician.
Contents[hide] |
[edit]Biography
Tukey was born in New Bedford, Massachusetts in 1915, and obtained a B.A. in 1936 and M.Sc.in 1937, in chemistry, from Brown University, before moving to Princeton University where he received a Ph.D. in mathematics.[2]
During World War II, Tukey worked at the Fire Control Research Office and collaborated withSamuel Wilks and William Cochran. After the war, he returned to Princeton, dividing his time between the university and AT&T Bell Laboratories.
Among many contributions to civil society, Tukey served on a committee of the American Statistical Association that produced a report challenging the conclusions of the Kinsey Report,Statistical Problems of the Kinsey Report on Sexual Behavior in the Human Male.
He was awarded the IEEE Medal of Honor in 1982 “For his contributions to the spectral analysis of random processes and the fast Fourier transform (FFT) algorithm.”
Tukey retired in 1985. He died in New Brunswick, New Jersey on July 26, 2000.
[edit]Scientific contributions
His statistical interests were many and varied. He is particularly remembered for his development with James Cooley of the Cooley–Tukey FFT algorithm. In 1970, he contributed significantly to what is today known as the jackknife estimation—also termed Quenouille-Tukey jackknife. He introduced the box plot in his 1977 book,”Exploratory Data Analysis“.
Tukey’s range test, the Tukey lambda distribution, Tukey’s test of additivity and Tukey’s lemma all bear his name. He is also the creator of several little-known methods such as the trimean andmedian-median line, an easier alternative to linear regression.
In 1974, he developed, with Jerome H. Friedman, the concept of the projection pursuit.[3]
http://en.wikipedia.org/wiki/Ronald_Fisher
Sir Ronald Aylmer Fisher FRS (17 February 1890 – 29 July 1962) was an English statistician,evolutionary biologist, eugenicist and geneticist. Among other things, Fisher is well known for his contributions to statistics by creating Fisher’s exact test and Fisher’s equation. Anders Hald called him “a genius who almost single-handedly created the foundations for modern statistical science”[1] while Richard Dawkins named him “the greatest biologist since Darwin“.[2]
http://en.wikipedia.org/wiki/William_Sealy_Gosset
William Sealy Gosset (June 13, 1876–October 16, 1937) is famous as a statistician, best known by his pen name Student and for his work on Student’s t-distribution.
Born in Canterbury, England to Agnes Sealy Vidal and Colonel Frederic Gosset, Gosset attendedWinchester College before reading chemistry and mathematics at New College, Oxford. On graduating in 1899, he joined the Dublin brewery of Arthur Guinness & Son.
Guinness was a progressive agro-chemical business and Gosset would apply his statistical knowledge both in the brewery and on the farm—to the selection of the best yielding varieties ofbarley. Gosset acquired that knowledge by study, trial and error and by spending two terms in 1906–7 in the biometric laboratory of Karl Pearson. Gosset and Pearson had a good relationship and Pearson helped Gosset with the mathematics of his papers. Pearson helped with the 1908 papers but he had little appreciation of their importance. The papers addressed the brewer’s concern with small samples, while the biometrician typically had hundreds of observations and saw no urgency in developing small-sample methods.
Another researcher at Guinness had previously published a paper containing trade secrets of the Guinness brewery. To prevent further disclosure of confidential information, Guinness prohibited its employees from publishing any papers regardless of the contained information. However, after pleading with the brewery and explaining that his mathematical and philosophical conclusions were of no possible practical use to competing brewers, he was allowed to publish them, but under a pseudonym (“Student”), to avoid difficulties with the rest of the staff.[1] Thus his most famous achievement is now referred to as Student’s t-distribution, which might otherwise have been Gosset’s t-distribution.
Top 25 Errors in Programming that lead to hacker attacks
I am elaborating an earlier article on https://decisionstats.com/top-25-most-dangerous-software-errors/ based on my continued research into cyber conflict and strategy. My inputs are in italics – the rest is a condensed article for further thought.
This is thus a very useful initiative for the world to follow and upgrade their cyber security.
It is in accordance with the US policy to secure its cyber infrastructure (http://www.whitehouse.gov/the-press-office/remarks-president-securing-our-nations-cyber-infrastructure) and countries like India, and even Europe as well as other nations could do well to atleast benchmark their own security practices in software and digital infrastructure with it. There seems to much better technical coordination between rogue hackers than patriotic hackers imho 😉
The Department of Homeland Security of the United States of America has just launched a list of top 25 errors in programming or creating software that increase vulnerability to hacking attacks. The list which is available at http://cwe.mitre.org/top25/index.html lists down a methodology fo measuring vulnerability called Common Weakness Scoring System (CWSS) and uses that score to rank the various errors as well as suggestions to eliminate these weaknesses or errors.
Measuring Weaknesses
The importance of a weakness (that arises due to software bugs) may vary depending on business usage or project implementation, the technologies , operating systems and computing environments in use, and the risk or threat perception.The Common Weakness Scoring System (CWSS) provides a mechanism for scoring weaknesses. and provides a framework for prioritizing security errors (“weaknesses”) that are discovered in software applications.
Identifying Weaknesses
For example the number 1 weakness is shown with
1CWE-89: Improper Neutralization of Special Elements used in an SQL Command (‘SQL Injection’).
The rest of the weaknesses are
RANK SCORE ID NAME
[1] 93.8 CWE-89 Improper Neutralization of Special Elements used in an SQL Command (‘SQL Injection’)
[2] 83.3 CWE-78 Improper Neutralization of Special Elements used in an OS Command (‘OS Command Injection’)
[3] 79.0 CWE-120 Buffer Copy without Checking Size of Input (‘Classic Buffer Overflow’)
[4] 77.7 CWE-79 Improper Neutralization of Input During Web Page Generation (‘Cross-site Scripting’)
[5] 76.9 CWE-306 Missing Authentication for Critical Function
[6] 76.8 CWE-862 Missing Authorization
[7] 75.0 CWE-798 Use of Hard-coded Credentials
[8] 75.0 CWE-311 Missing Encryption of Sensitive Data
[9] 74.0 CWE-434 Unrestricted Upload of File with Dangerous Type
[10] 73.8 CWE-807 Reliance on Untrusted Inputs in a Security Decision
[11] 73.1 CWE-250 Execution with Unnecessary Privileges
[12] 70.1 CWE-352 Cross-Site Request Forgery (CSRF)
[13] 69.3 CWE-22 Improper Limitation of a Pathname to a Restricted Directory (‘Path Traversal’)
[14] 68.5 CWE-494 Download of Code Without Integrity Check
[15] 67.8 CWE-863 Incorrect Authorization
[16] 66.0 CWE-829 Inclusion of Functionality from Untrusted Control Sphere
[17] 65.5 CWE-732 Incorrect Permission Assignment for Critical Resource
[18] 64.6 CWE-676 Use of Potentially Dangerous Function
[19] 64.1 CWE-327 Use of a Broken or Risky Cryptographic Algorithm
[20] 62.4 CWE-131 Incorrect Calculation of Buffer Size
[21] 61.5 CWE-307 Improper Restriction of Excessive Authentication Attempts
[22] 61.1 CWE-601 URL Redirection to Untrusted Site (‘Open Redirect’)
[23] 61.0 CWE-134 Uncontrolled Format String
[24] 60.3 CWE-190 Integer Overflow or Wraparound
[25] 59.9 CWE-759 Use of a One-Way Hash without a Salt
Details of each weakness is given by http://cwe.mitre.org/top25/index.html#Details
It includes Summary , Weakness Prevalence, Consequences, Remediation Cost, Ease of Detection ,Attacker Awareness and Attack Frequency .In addition the following sections describe each software vulnerability in detail- Technical Details ,Code Examples ,Detection Methods ,References,Prevention and Mitigation, Related CWEs and Related Attack Patterns.
Other important software weaknesses are –
[26] CWE-770: Allocation of Resources Without Limits or Throttling
[27] CWE-129: Improper Validation of Array Index
[28] CWE-754: Improper Check for Unusual or Exceptional Conditions
[29] CWE-805: Buffer Access with Incorrect Length Value
[30] CWE-838: Inappropriate Encoding for Output Context
[31] CWE-330: Use of Insufficiently Random Values
[32] CWE-822: Untrusted Pointer Dereference
[33] CWE-362: Concurrent Execution using Shared Resource with Improper Synchronization (‘Race Condition’)
[34] CWE-212: Improper Cross-boundary Removal of Sensitive Data
[35] CWE-681: Incorrect Conversion between Numeric Types
[36] CWE-476: NULL Pointer Dereference
[37] CWE-841: Improper Enforcement of Behavioral Workflow
[38] CWE-772: Missing Release of Resource after Effective Lifetime
[39] CWE-209: Information Exposure Through an Error Message
[40] CWE-825: Expired Pointer Dereference
[41] CWE-456: Missing Initialization
Mitigating Weaknesses
Here is an example of the new matrix for migrations that also list the top 25 errors . This thus shows a way to fix the weaknesses and relative impact on each weakness by the following mitigations.
http://cwe.mitre.org/top25/mitigations.html#MitigationMatrix
Effectiveness ratings include:
- High: The mitigation has well-known, well-understood strengths and limitations; there is good coverage with respect to variations of the weakness.
- Moderate: The mitigation will prevent the weakness in multiple forms, but it does not have complete coverage of the weakness.
- Limited: The mitigation may be useful in limited circumstances, only be applicable to a subset of this weakness type, require extensive training/customization, or give limited visibility.
- Defense in Depth (DiD): The mitigation may not necessarily prevent the weakness, but it may help to minimize the potential impact when an attacker exploits the weakness.
Within the matrix, the following mitigations are identified:
- M1: Establish and maintain control over all of your inputs.
- M2: Establish and maintain control over all of your outputs.
- M3: Lock down your environment.
- M4: Assume that external components can be subverted, and your code can be read by anyone.
- M5: Use industry-accepted security features instead of inventing your own.
The following general practices are omitted from the matrix:
- GP1: Use libraries and frameworks that make it easier to avoid introducing weaknesses.
- GP2: Integrate security into the entire software development lifecycle.
- GP3: Use a broad mix of methods to comprehensively find and prevent weaknesses.
- GP4: Allow locked-down clients to interact with your software.
| M1 | M2 | M3 | M4 | M5 | CWE |
|---|---|---|---|---|---|
| High | DiD | Mod | CWE-22: Improper Limitation of a Pathname to a Restricted Directory (‘Path Traversal’) | ||
| Mod | High | DiD | Ltd | CWE-78: Improper Neutralization of Special Elements used in an OS Command (‘OS Command Injection’) | |
| Mod | High | Ltd | CWE-79: Improper Neutralization of Input During Web Page Generation (‘Cross-site Scripting’) | ||
| Mod | High | DiD | Ltd | CWE-89: Improper Neutralization of Special Elements used in an SQL Command (‘SQL Injection’) | |
| Mod | DiD | Ltd | CWE-120: Buffer Copy without Checking Size of Input (‘Classic Buffer Overflow’) | ||
| Mod | DiD | Ltd | CWE-131: Incorrect Calculation of Buffer Size | ||
| High | DiD | Mod | CWE-134: Uncontrolled Format String | ||
| Mod | DiD | Ltd | CWE-190: Integer Overflow or Wraparound | ||
| High | CWE-250: Execution with Unnecessary Privileges | ||||
| Mod | Mod | CWE-306: Missing Authentication for Critical Function | |||
| Mod | CWE-307: Improper Restriction of Excessive Authentication Attempts | ||||
| DiD | CWE-311: Missing Encryption of Sensitive Data | ||||
| High | CWE-327: Use of a Broken or Risky Cryptographic Algorithm | ||||
| Ltd | CWE-352: Cross-Site Request Forgery (CSRF) | ||||
| Mod | DiD | Mod | CWE-434: Unrestricted Upload of File with Dangerous Type | ||
| DiD | CWE-494: Download of Code Without Integrity Check | ||||
| Mod | Mod | Ltd | CWE-601: URL Redirection to Untrusted Site (‘Open Redirect’) | ||
| Mod | High | DiD | CWE-676: Use of Potentially Dangerous Function | ||
| Ltd | DiD | Mod | CWE-732: Incorrect Permission Assignment for Critical Resource | ||
| High | CWE-759: Use of a One-Way Hash without a Salt | ||||
| DiD | High | Mod | CWE-798: Use of Hard-coded Credentials | ||
| Mod | DiD | Mod | Mod | CWE-807: Reliance on Untrusted Inputs in a Security Decision | |
| High | High | High | CWE-829: Inclusion of Functionality from Untrusted Control Sphere | ||
| DiD | Mod | Mod | CWE-862: Missing Authorization | ||
| DiD | Mod | CWE-863: Incorrect Authorization |
Lovely forecasting blog
I really loved this simple, smart and yet elegant explanation of forecasting. even a high school quarterback could understand it, and maybe get a internship job building and running and re running code for Mars shot.
Despite my plea that you remain svelte in real life, I implore you to be naïve in business forecasting – and use a naïve forecasting model early and often. A naïve forecasting model is the most important model you will ever use in business forecasting.
and now the killer line
Purists may argue that the only true naïve forecast is the “no-change” forecast, meaning either a random walk (forecast = last known actual) or a seasonal random walk (e.g. forecast = actual from corresponding period last year). These are referred to as NF1 and NF2 in the Makridakis text (where NF = Naïve Forecast). In our 2006 SAS webseries Finding Flaws in Forecasting, an attendee asked “What about using a simple time series forecast with no intervention as the naïve forecast?” Is that allowed?
i did write a blog article on forecasting some time back, but back then I was a little blogger, with the website name being http://iwannacrib.com
great work in helping make forecasting easier to understand for people who have flower shops and dont have a bee, to help them with the forecasts, nor an geeky email list, not 4000$.
make it easier for the little guy to forecast his sales, so he cuts down on his supply chain inventory, lowering his carbon footprint.
Blog.sas.com take a bow, on labour day, helping workers with easy to understand models.
http://blogs.sas.com/forecasting/index.php?/archives/68-Which-Naive-Model-to-Use.html
Getting Flatr on a wordpress.com blog
What is Flattr?
social micro payments- aka another way for bloggers, tweeters, facebookies to make money.
Thing of it as the Paypal plus a ReTweetmeme button.
FlattR is the new legal business of the creator of Pirate Bay- the large search engine for bit torrent data.
and how to enable it on WordPress.com
Read some snarkly grrovy instructions here with a screenshot
1.) Open a Flattr.com account here. This should be reasonably straightforward. A monkey hitting keys at random could manage it in about half an hour. It took me less than 45 minutes.
2.) In the top right of ‘Your Flattr Dashboard’ there is a button ‘Submit Thing’. Click on that and enter the details of your blog – the URL (like decisionstats.com for me) and a description (make that atleast 3 sentences). Flattr will create a page – for example,https://flattr.com/thing/162940/example-blog
Now go to your wordpress dashboard- sharing tab.
/wp-admin/options-general.php?page=sharing
Add the following lines to your New Add Service in respective tabs
URL= https://flattr.com/thing/175763/DecisionStats (change this to the one created for yourself instep2 above)
ICON = http://api.flatrr.com/button/flattr-badge-large.png
If you have a non WordPress blog see instructions at http://markup.io/v/jz3wv155bsfg or screenshot of instructions here-
Is Random Poetry Click Fraud
Is poetry when randomized
Tweaked, meta tagged , search engine optimized
Violative of unseen terms and conditional clauses
Is random poetry or aggregated prose farmed for click fraud uses
I dont know, you tell me, says the blog boy,
Tapping away at the keyboard like a shiny new toy,
Geeks unfortunately too often are men too many,
Forgive the generalization, but the tech world is yet to be equalized.
If a New York Hot Dog is a slice of heaven at four bucks a piece
Then why is prose and poetry at five bucks an hour considered waste
Ah I see, you have grown old and cynical,
Of the numerous stupid internet capers and cyber ways
The clicking finger clicks on
swiftly but mostly delightfully virally moves on
While people collect its trails and
ponder its aggregated merry ways
All people are equal but all links are not,
Thus overturning two centuries of psychology had you been better taught,
But you chose to drop out of school, and create that search engine so big
It is now a fraud catchers head ache that millions try to search engine optimize and rig
Once again, people are different, in so many ways so prettier
Links are the same hyper linked code number five or earlier
People think like artificial artificial (thus natural) neural nets
Biochemically enhanced Harmonically possessed.
rather than analyze forensically and quite creepily
where people have been
Gentic Algorithms need some chaos
To see what till now hasnt been seen.
Again this was a random poem,
inspired by a random link that someone clicked
To get here, on a carbon burning cyber machine,
Having digested poem, moves on, unheard , unseen.
Related Articles
(Inspired by the Hyper Link at http://goo.gl/a8ijW )
Also-
- Getting to the Bottom of Facebook Click Fraud (actionableinsights.covario.com)
- Click Fraud: A Legal Look (firstrate.co.nz)
- Microsoft says Google used click fraud to orchestrate Bing Sting (gabriellahiresit.com)
- Vile poetry virtuous poetry for which my mind aches and soul groans lol (ask.metafilter.com)
- Click fraud on the rise again? (silvertailsystems.wordpress.com)
- Study: Click Fraud Drops Slightly in Q4 (pcworld.com)
- Click Fraud: What Is Your Ad Network Doing About It? (searchenginewatch.com)
Open Source Compiler for SAS language/ GNU -DAP
I am still testing this out.
But if you know bit more about make and .compile in Ubuntu check out
http://www.gnu.org/software/dap/
I loved the humorous introduction
Dap is a small statistics and graphics package based on C. Version 3.0 and later of Dap can read SBS programs (based on the utterly famous, industry standard statistics system with similar initials – you know the one I mean)! The user wishing to perform basic statistical analyses is now freed from learning and using C syntax for straightforward tasks, while retaining access to the C-style graphics and statistics features provided by the original implementation. Dap provides core methods of data management, analysis, and graphics that are commonly used in statistical consulting practice (univariate statistics, correlations and regression, ANOVA, categorical data analysis, logistic regression, and nonparametric analyses).
Anyone familiar with the basic syntax of C programs can learn to use the C-style features of Dap quickly and easily from the manual and the examples contained in it; advanced features of C are not necessary, although they are available. (The manual contains a brief introduction to the C syntax needed for Dap.) Because Dap processes files one line at a time, rather than reading entire files into memory, it can be, and has been, used on data sets that have very many lines and/or very many variables.
I wrote Dap to use in my statistical consulting practice because the aforementioned utterly famous, industry standard statistics system is (or at least was) not available on GNU/Linux and costs a bundle every year under a lease arrangement. And now you can run programs written for that system directly on Dap! I was generally happy with that system, except for the graphics, which are all but impossible to use, but there were a number of clumsy constructs left over from its ancient origins.
http://www.gnu.org/software/dap/#Sample output
Copyright © 2001, 2002, 2003, 2004 Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA
sounds too good to be true- GNU /DAP joins WPS workbench and Dulles Open’s Carolina as the third SAS language compiler (besides the now defunct BASS software) see http://en.wikipedia.org/wiki/SAS_language#Controversy
Also see http://en.wikipedia.org/wiki/DAP_(software)
Dap was written to be a free replacement for SAS, but users are assumed to have a basic familiarity with the C programming language in order to permit greater flexibility. Unlike R it has been designed to be used on large data sets.
It has been designed so as to cope with very large data sets; even when the size of the data exceeds the size of the computer’s memory
Related Articles
- R courses from Statistics.com (revolutionanalytics.com)
- Categorical Data Analysis for the Behavioral and Social Sciences (psypress.com)
- Skills of a good data miner (zyxo.wordpress.com)
- GNU Octave 3.4 has just been released (gnu.org)
- Revolution R Enterprise 4.2 now available (revolutionanalytics.com)
DECISION STATS 