Home » Posts tagged 'server'
Tag Archives: server
Decisionstats.com is back from a dDOS
- Servers were okay, it was the DNS server that got swamped.
- I am sorry for the downtime- hopefully you didnt even notice
- I have faced challenges like domain name hijacking, sql injection , malicious WP plugins and thats why shifted to a professional hosting. I stand by my vendors and their professional judgement, moving away would mean the hackers won.
- This was very clever to swamp the DNS provider- my compliments to the tech talent behind this.
- You would think that every webmaster would have a back up plan in case his site went dDOS, but surprisingly even corporate websites dont have a back up (under attack) plan
Latest R Journal
Including juicy stuff on using a cluster of Apple Machines for grid computing , seasonality forecasting (Yet Another Package For Time Series )
But I kind of liked Sumo too-
https://code.google.com/p/sumo/
Sumo is a fully-functional web application template that exposes an authenticated user’s R session within java server pages.
Sumo: An Authenticating Web Application with an Embedded R Session by Timothy T. Bergsma and Michael S. Smith Abstract Sumo is a web application intended as a template for developers. It is distributed as a Java ‘war’ file that deploys automatically when placed in a Servlet container’s ‘webapps’
directory. If a user supplies proper credentials, Sumo creates a session-specific Secure Shell connection to the host and a user-specific R session over that connection. Developers may write dynamic server pages that make use of the persistent R session and user-specific file space.
and for Apple fanboys-
We created the xgrid package (Horton and Anoke, 2012) to provide a simple interface to this distributed computing system. The package facilitates use of an Apple Xgrid for distributed processing of a simulation with many independent repetitions, by simplifying job submission (or grid stuffing) and collation of results. It provides a relatively thin but useful layer between R and Apple’s ‘xgrid’ shell command, where the user constructs input scripts to be run remotely. A similar set of routines, optimized for parallel estimation of JAGS (just another Gibbs sampler) models is available within the runjags package (Denwood, 2010). However, with the exception of runjags, none of the previously mentioned packages support parallel computation over an Apple Xgrid.
Hmm I guess parallel computing enabled by Wifi on mobile phones would be awesome too ! So would be anything using iOS . See the rest of the R Journal at http://journal.r-project.org/current.html
The economics of software piracy
Software piracy exists because-
1) Lack of appropriate technological controls (like those on DVDs) or on Bit Torrents (an innovation on the centralized server like Napster) or on Streaming etc etc.
Technology to share content has evolved at a much higher pace than technology to restrict content from being shared or limited to purchasers.
2) Huge difference in purchasing power across the globe.
An Itunes song at 99 cents might be okay buy in USA, but in Asia it is very expensive. Maybe if content creators use Purchasing Power Parity to price their goods, it might make an indent.
3) State sponsored intellectual theft as another form of economic warfare- this has been going on since the West stole gunpowder and silk from the Chinese, and Intel decided to win back the IP rights to the microprocessor (from the Japanese client)
4) Lack of consensus in policy makers across the globe on who gets hurt from IP theft, but complete consensus across young people in the globe that they are doing the right thing by downloading stuff for free.
5) There is no such thing as a free lunch. Sometimes software (and movie and songs) piracy help create demand across ignored markets – I always think the NFL can be huge in India if they market it.Sometimes it forces artists to commit suicide because they give up on the life of starving musician.
Mostly piracy has helped break profits of intermediaries between the actual creator and actual consumer.
So how to solve software piracy , assuming it is something that can be solved-
I dont know, but I do care.
I give most of my writings as CC-by-SA and that includes my poems. People (friends and family) sometimes pay me not to sing.
Pirates have existed and will exist as long as civilized men romanticize the notion of piracy and bicker between themselves for narrow gains.
- Ephesians 4:28 Let the thief no longer steal, but rather let him labor, doing honest work with his own hands, so that he may have something to share with anyone in need.
- A clean confession, combined with a promise never to commit the sin again, when offered before one who has the right to receive it, is the purest type of repentance.-Gandhi
- If you steal, I will wash your mouth with soap- Anonymous Mother.
- You shall not steal- Moses
- Steal may refer to: Theft, the illegal taking of another person’s property without that person’s freely-given consent; The gaining of a stolen base in baseball;
Working with a large number of files for reading into R #rstats
Using the dir() and list.files() commands lists all the files in a particular directory. These can be interactively read by R, by referencing to specific parts of the list created by the above two commands. This is useful when you are working with a large number of files, that get generated or re-generated after specific time periods (like web server log files)
> getwd()
[1] “C:/Users/KUs/Documents”
> path=”C:/Users/KUs/Desktop/tester”
> dir(path)
[1] “tester.csv” “tester2.csv” “tester3.csv”"tester4.csv”
> setwd(path)
> read.table(file=dir(path)[1],sep=”t”,header=T)
X1 X2 X3 X4
1 to be 2 B
> read.table(file=dir(path)[4],sep=”,”,header=T)
zoo bee doo bee.1 daa
1 12 32 43 34 qwerty
Revolution R Enterprise 6.0 launched!
Just got the email-more software is good news!
Revolution R Enterprise 6.0 for 32-bit and 64-bit Windows and 64-bit Red Hat Enterprise Linux (RHEL 5.x and RHEL 6.x) features an updated release of the RevoScaleR package that provides fast, scalable data management and data analysis: the same code scales from data frames to local, high-performance .xdf files to data distributed across a Windows HPC Server cluster or IBM Platform Computing LSF cluster. RevoScaleR also allows distribution of the execution of essentially any R function across cores and nodes, delivering the results back to the user.
Detailed information on what’s new in 6.0 and known issues:
http://www.revolutionanalytics.com/doc/README_RevoEnt_Windows_6.0.0.pdf
and from the manual-lots of function goodies for Big Data
- IBM Platform LSF Cluster support [Linux only]. The new RevoScaleR function, RxLsfCluster, allows you to create a distributed compute context for the Platform LSF workload manager.
- Azure Burst support added for Microsoft HPC Server [Windows only]. The new RevoScaleR function, RxAzureBurst, allows you to create a distributed compute context to have computations performed in the cloud using Azure Burst
- The rxExec function allows distributed execution of essentially any R function across cores and nodes, delivering the results back to the user.
- functions RxLocalParallel and RxLocalSeq allow you to create compute context objects for local parallel and local sequential computation, respectively.
- RxForeachDoPar allows you to create a compute context using the currently registered foreach parallel backend (doParallel, doSNOW, doMC, etc.). To execute rxExec calls, simply register the parallel backend as usual, then set your compute context as follows: rxSetComputeContext(RxForeachDoPar())
- rxSetComputeContext and rxGetComputeContext simplify management of compute contexts.
- rxGlm, provides a fast, scalable, distributable implementation of generalized linear models. This expands the list of full-featured high performance analytics functions already available: summary statistics (rxSummary), cubes and cross tabs (rxCube,rxCrossTabs), linear models (rxLinMod), covariance and correlation matrices (rxCovCor),
binomial logistic regression (rxLogit), and k-means clustering (rxKmeans)example: a Tweedie family with 1 million observations and 78 estimated coefficients (categorical data)
took 17 seconds with rxGlm compared with 377 seconds for glm on a quadcore laptopand easier working with R’s big brother SAS language
RevoScaleR high-performance analysis functions will now conveniently work directly with a variety of external data sources (delimited and fixed format text files, SAS files, SPSS files, and ODBC data connections). New functions are provided to create data source objects to represent these data sources (RxTextData, RxOdbcData, RxSasData, and RxSpssData), which in turn can be specified for the ‘data’ argument for these RevoScaleR analysis functions: rxHistogram, rxSummary, rxCube, rxCrossTabs, rxLinMod, rxCovCor, rxLogit, and rxGlm.
example,
you can analyze a SAS file directly as follows:
# Create a SAS data source with information about variables and # rows to read in each chunk
sasDataFile <- file.path(rxGetOption(“sampleDataDir”),”claims.sas7bdat”)
sasDS <- RxSasData(sasDataFile, stringsAsFactors = TRUE,colClasses = c(RowNum = “integer”),rowsPerRead = 50)# Compute and draw a histogram directly from the SAS file
rxHistogram( ~cost|type, data = sasDS)
# Compute summary statistics
rxSummary(~., data = sasDS)
# Estimate a linear model
linModObj <- rxLinMod(cost~age + car_age + type, data = sasDS)
summary(linModObj)
# Import a subset into a data frame for further inspection
subData <- rxImport(inData = sasDS, rowSelection = cost > 400,
varsToKeep = c(“cost”, “age”, “type”))
subData
The installation instructions and instructions for getting started with Revolution R Enterprise & RevoDeployR for Windows: http://www.revolutionanalytics.com/downloads/instructions/windows.php
Anonymous Operation India- Using Amazon AWS to go to PirateBay
The cyber -group known as Anonymous has now decided to fight for internet freedom for my 1.2 billion countrymen (India)
So in operation India they go and knock some websites off. The immediate provocation-
1) Legal System prevented access to Pirate Bay (and other sites)
This as per Anons restricts the freedom of glorious motherland of India (which incidentally does have a high number of engineers).
A slight modification to using violence (like DDOS) is to use non violence-this approach is use the free tier at Amazon EC2-http://aws.amazon.com/free/ and sign up and start the windows tier
AWS Free Usage Tier (Per Month): ( only if your torrents are going to be less than 15 gb a month!!)
- 750 hours of Amazon EC2 Linux Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month
* - 750 hours of Amazon EC2 Microsoft Windows Server Micro Instance usage (613 MB of memory and 32-bit and 64-bit platform support) – enough hours to run continuously each month
* - 750 hours of an Elastic Load Balancer plus 15 GB data processing*
- 30 GB of Amazon Elastic Block Storage, plus 2 million I/Os and 1 GB of snapshot storage
* - 5 GB of Amazon S3 standard storage, 20,000 Get Requests, and 2,000 Put Requests
* - 100 MB of storage, 5 units of write capacity, and 10 units of read capacity for Amazon DynamoDB.**
- 25 Amazon SimpleDB Machine Hours and 1 GB of Storage
** - 1,000 Amazon SWF workflow executions can be initiated for free. A total of 10,000 activity tasks, signals, timers and markers, and 30,000 workflow-days can also be used for free
** - 100,000 Requests of Amazon Simple Queue Service
** - 100,000 Requests, 100,000 HTTP notifications and 1,000 email notifications for Amazon Simple Notification Service
** - 10 Amazon Cloudwatch metrics, 10 alarms, and 1,000,000 API requests
** - 15 GB of bandwidth out aggregated across all AWS services
*
and get download speeds of 190 kb/ps to connect to Pirate Bay from the US !!
So you dont know Linux, huh (but do know how to Torrent). Well Amazon has a Windows instance for free too. Shame on you for not knowing Linux though! Illegal torrents hurt artists like Shahrukh Khan the most!!!
http://aws.amazon.com/windows/
How to create a Windows Amazon Instance
http://aws.amazon.com/resources/webinars/?vid=OLfmqcYnhUM&p=015041767CFA57C8
and to download your precious data (why?) from your remote instance to your local PC use these instructions.
1. Go to find the RDP file amazon asked you to downloaded onto your local PC. right-click –> Edit
2. Go to “Local Resources” tab –> “Local devices and resources” –> “More” button
3. Expand the “Drives” and check the disks you want to share when you TS to the remote box.
4. after connect, you will see the new drives in My Computer already mounted for you.
For me, copy speed is 200-300kB/Second. Enjoy!
or even easier
Installing dropbox on both your client machine and EC2 instance is one of the easiest ways to do it. (go to http://dropbox.com) or try the new Google Drive to share content.
–
As for Anonymous- DDOS attacks are easy, IRC press conferences are fun, but there are enough techies in India ,kids.
NOTE- You are liable legally for your actions whether on Amazon AWS or on your own laptop. This is just a technical note- not a moral note.
PS- I wonder if the Chinese can use this to access Facebook. Maybe it is time Anonymous got the guts to hit China for it’s unfree internet.
PPS- Message to Anons— Next time, try giving us a pdf tutorial on how to create an anonymized sql injection/ddos !
Custom T Shirt-
INDIA- Writing code since 3000 BC.
INDIA- We made the zero 0.
Oracle R Updated!
Interesting message from https://blogs.oracle.com/R/ the latest R blog
_——–_
Oracle just released the latest update to Oracle R Enterprise, version 1.1. This release includes the Oracle R Distribution (based on open source R, version 2.13.2), an improved server installation, and much more. The key new features include:
- Extended Server Support: New support for Windows 32 and 64-bit server components, as well as continuing support for Linux 64-bit server components
- Improved Installation: Linux 64-bit server installation now provides robust status updates and prerequisite checks
- Performance Improvements: Improved performance for embedded R script execution calculations
In addition, the updated ROracle package, which is used with Oracle R Enterprise, now reads date data by conversion to character strings.
We encourage you download Oracle software for evaluation from the Oracle Technology Network. See these links for R-related software: Oracle R Distribution, Oracle R Enterprise, ROracle, Oracle R Connector for Hadoop. As always, we welcome comments and questions on the Oracle R Forum.
Oracle R Distribution 2-13.2 Update Available
Oracle has released an update to the Oracle R Distribution, an Oracle-supported distribution of open source R. Oracle R Distribution 2-13.2 now contains the ability to dynamically link the following libraries on both Windows and Linux:
- The Intel Math Kernel Library (MKL) on Intel chips
- The AMD Core Math Library (ACML) on AMD chips
To take advantage of the performance enhancements provided by Intel MKL or AMD ACML in Oracle R Distribution, simply add the MKL or ACML shared library directory to the LD_LIBRARY_PATH system environment variable. This automatically enables MKL or ACML to make use of all available processors, vastly speeding up linear algebra computations and eliminating the need to recompile R. Even on a single core, the optimized algorithms in the Intel MKL libraries are faster than using R’s standard BLAS library.
Open-source R is linked to NetLib’s BLAS libraries, but they are not multi-threaded and only use one core. While R’s internal BLAS are efficient for most computations, it’s possible to recompile R to link to a different, multi-threaded BLAS library to improve performance on eligible calculations. Compiling and linking to R yourself can be involved, but for many, the significantly improved calculation speed justifies the effort. Oracle R Distribution notably simplifies the process of using external math libraries by enabling R to auto-load MKL orACML. For R commands that don’t link to BLAS code, taking advantage of database parallelism usingembedded R execution in Oracle R Enterprise is the route to improved performance.
For more information about rebuilding R with different BLAS libraries, see the linear algebra section in the R Installation and Administration manual. As always, the Oracle R Distribution is available as a free download to anyone. Questions and comments are welcome on the Oracle R Forum.
