Show random products in Magento: You are doing it wrong

Well, probably (I certainly have).

A typical requirement is to show random products on the home page, random products of a category on the category page, or similar. Magento offers a simple block that can show a specified number of random products, Mage_Catalog_Block_Product_List_Random. To add filtering, like by category, you still need to write your own block, but it could easily be extended.

This is the CMS code:

{{block type="catalog/product_list_random" num_products="4"}}

But don’t get too excited. This block type will make your site slow, you should not use it. The problem is in this line:

$collection->getSelect()->order('rand()');

I’ve been recently advocating against using ORDER BY RAND() to select random products from a Magento database. 1 Generally, this method is discouraged because of its performance issues.

Continue reading “Show random products in Magento: You are doing it wrong”

Notes:

  1. For example in this Magento StackExchange post: Four Random Products on Homepage

Efficiently Draw Random Elements From Large PHP Array

I recently needed to select few random elements from a big database table and was looking for alternatives to ORDER BY RAND() (because of its performance issues). Because the IDs are not continuous, the proposed solutions in the linked article are not sufficient.

So the idea is to fetch all IDs from the table, pick X random IDs and query these directly. The memory overhead of reading an array of about 100,000 IDs is not too big, so the problem is reduced to how to pick random IDs efficiently with PHP.
Continue reading “Efficiently Draw Random Elements From Large PHP Array”

Comparable Interface For PHP

About 5-6 years ago I had my “PHP should be more like Java” phase and experimented a lot with things like string objects and method overloading, which usually required hackish workarounds and most things did not turn out to be very practical in the long run.

But there is one package I still like very much, namely ComparatorTools, which got to be place 2 in the monthly PHPclasses.org innovation awards after all. It provides Comparable and Comparator interfaces and functions similar to the core array functions, that can work with these.

Interfaces

The interfaces resemble the corresponding Java interfaces, except that we do not have Generics in PHP, so it is cannot be guaranteed that compared objects have the same type. This has to be checked at runtime in the implementation, if needed. An exception type for these cases is provided:

interface Comparable
{
	/**
	 * @param object $object
	 * @return numeric negative value if $this < $object, positive if $this > $object, 0 otherwise (if objects are considered equal)
	 * @throws ComparatorException if objects are not comparable to each other
	 */
	public function compareTo($object);
}
interface Comparator
{
	/**
	 * @param object $object1
	 * @param object $object2
	 * @return numeric Negative value if $object1 < $object2, positive if $object1 > $object2, 0 otherwise
	 * @throws ComparatorException if objects are not comparable to each other
	 */
	public function compare($object1, $object2);
}

Continue reading “Comparable Interface For PHP”

CSV Processing In Magento

A development principle , not only with Magento, is that you shouldn’t try to reinvent the wheel and especially use the functions of the used framework wherever possible. Magento has many more or less known universal helpers, in the helper classes in Mage_Core as well as in lib/Varien and of course in the Zend Framework.

A classic is for example JSON encoding. Although PHP has its built-in functions json_encode and json_decode, but they have some shortcomings that are compensated for in the Zend_Json implementation. So Zend_Json::encode() has a cycle check, Magento added support for inline translations within JSON strings in Mage_Core_Helper_Data::jsonEncode(). Thus in Magento you always should use Mage::helper('core')->jsonEncode() (and jsonDecode).

Varien_File_Csv

How is it with processing CSV files? Since import and export works with CSV files in the standard implementation, Magento should have somthing, right? Presenting Varien_File_Csv! Well, I’ll anticipate the result: except for very simple tasks with small files, the class is not useful at all.

Continue reading “CSV Processing In Magento”

A JSON-RPC Adapter For The Magento API

Going through my old answers on Magento StackExchange, I found this question about using the Magento API via JavaScript and noticed that the link to GitHub that contained an important part of the solution, namely implementing a JSON-RPC adapter for the Magento-API is dead now.

So I decided to publish my complete module myself (the original link was a core hack, I rewrote it as a clean module):

GitHub: SGH_JsonRpc

The whole module is less than 100 lines of code. In config.xml our controller is added to the api route:

    <frontend>
        <routers>
            <api>
                <args>
                    <modules>
                        <sgh_jsonrpc before="Mage_Api">SGH_JsonRpc_Api</sgh_jsonrpc>
                    </modules>
                </args>
            </api>
        </routers>
    </frontend>

The new API adapter is defined in api.xml:

Continue reading “A JSON-RPC Adapter For The Magento API”

Efficiently Increase/Decrease Magento Attributes

Magento.SE Screenshot

This question arose on Magento StackExchange:

I need to decrement a value with an atomic database operation, is it possible using Magento models?

It is in fact possible, with a lesser known technique using Zend_Db_Expr. I’ll share it here as well:

$object->setNumber(new Zend_Db_Expr('number-1'));

For reference:

The method Mage_Core_Model_Resource_Abstract::_prepareDataForSave() contains the following code:

if ($object->hasData($field)) {
    $fieldValue = $object->getData($field);
    if ($fieldValue instanceof Zend_Db_Expr) {
        $data[$field] = $fieldValue;
    } else {
        ... [normal value processing follows]

EAV Models:

Note that you only can reference the attribute by its name (“number” in the example) if it’s a real column of the main table, not an EAV attribute.

Although the abovementioned method is only used by models with flat tables, Zend_Db_Expr can be used for EAV attributes as well, the method that handles it is Varien_Db_Adapter_Pdo_Mysql::prepareColumnValue().

BUT you always have use the column name “value“:

$product->setNumber(new Zend_Db_Expr('value-1'));

You don’t need to specify a table alias because during save each attribute gets processed with its own query, so “value” is not ambiguous.

Spryker vs. Magento

Recently I had the opportunity to take a peek into the source code of Spryker, the e-commerce framework that’s composing itself to be the new player in the enterprise area. Spryker was originally developed by the Berlin based incubator Project A to be used in the companies that they build up and is supposed to be released to the public this year. While “public” is not enirely true, I can say so much in advance: An open source version is not planned, licenses will cost about 100.000 € / year. Agencies can register as partner, to get access to source code and documentation without a license. With the first sold license they get access to additional training material. Freelancers are not addressed. Already listed as partner are the agencies CommercePlus and Symmetrics that are well-known in the German Magento scene.

What is special about Spryker?

Spryker doesn’t consider itself as ready-to-use product but as framework that provides the building blocks for an individual e-commerce solution from which you can choose and extend for your project. This takes the reality into account that no project is equal to the next and each shop has its own processes and infrastructure, which you have to address individually.

The core of Spryker are two separate applications, Yves and Zed. In short, Yves is a lightweight application for the frontend, Zed the big gun for the backend.

Discover Spryker
Image: http://spryker.com/product/

Yves (in the first version developed with Yii, now with Silex) reads any needed data from an in-memory NoSQL backend such as Redis.
Zed (Zend Framework 2) handles communication with MySQL, message queue and external systems, and contains the business logic for order processes and so on.

Read more at magenticians.com Continue reading “Spryker vs. Magento”

Magento Tutorial: How to Use Increment Models to Generate IDs (or SKUs)

Did you ever wonder how Magento generates the increment_id values for orders, invoices etc. and how to use or extend this mechanism? Maybe you found the eav_entity_store table which contains the last increment id per entity type and store and possibly a different prefix per store:

mysql> select * from eav_entity_store;
+-----------------+----------------+----------+------------------+-------------------+
| entity_store_id | entity_type_id | store_id | increment_prefix | increment_last_id |
+-----------------+----------------+----------+------------------+-------------------+
|               1 |              5 |        1 | 1                | 100000090         |
|               2 |              6 |        1 | 1                | 100000050         |
|               3 |              8 |        1 | 1                | 100000027         |
|               4 |              7 |        1 | 1                | 100000005         |
|               5 |              1 |        0 | 0                | 000000011         |
|               6 |              5 |        2 | 2                | 200000001         |
|               7 |              5 |        3 | 3                | 300000002         |
|               8 |              8 |        3 | 3                | 300000001         |
|               9 |              6 |        3 | 3                | 300000001         |
+-----------------+----------------+----------+------------------+-------------------+
9 rows in set (0.00 sec)

I will explain how to use this system in other entities in this article.

First of all, the standard method only works with EAV entities and only one attribute per entity can use the increment model and its name must be increment_id. I will explain later, how to overcome these limitations.

Standard Usage

The increment_id attribute should have the backend model eav/entity_attribute_backend_increment and the entity itself needs some additional configuration;

Entity Setup

Example: Setup Script
$installer->installEntities(array(
            'your_entity' => array(
                'entity_model' => 'your/entity_resource',
                'table' => 'your_entity_table',
                'increment_model' => 'eav/entity_increment_numeric',
                'increment_per_store' => 0,
                'increment_pad_length' => 8,
                'increment_pad_char' => '0',
                'attributes' => array(
                    'increment_id', array(
                        'type'      => Varien_Db_Ddl_Table::TYPE_TEXT,
                        'backend'   => 'eav/entity_attribute_backend_increment,
                    ),
                    // ... other attributes
                ),
            ),
        ));

Let’s have a look at the relevant fields:

  • increment_model: The model that is responsible for generating the increment ids. Either one of eav/entity_increment_numeric and eav/entity_increment_alphanum or a custom model. More about custom increment models below.
  • increment_per_store: (default: 0) 0 = the increment id is global for this entity, 1 = the increment id is incremented per store. You can set up a different prefix per store (see below)
  • increment_pad_length: (default: 8) the minimum length of the id. Shorter ids will be left padded with increment_pad_char (default: 0).
Notes
  1. If you want to save the increment id as a field in the main table, use static as type instead.
  2. For existing entities you can use updateEntityType instead of installEntityType

Prefix per Store

If you set “increment_per_store” to “1” in the entity setup, the increment ids get prefixed with the store_id by default, if you set it to “0” (global), they get prefixed with “0”. To set up different prefixes, use the Mage_Eav_Model_Entity_Store model. The corresponding database table eav_entity_store shown above gets automatically filled with one entry per entity and store if the entity has “increment_per_store” set, otherwise with only one entry per entity with store_id 0.
The table contains the prefix as well as the last increment id (which both should be used by the increment model to determine the next id).

Example: Set last id and prefix for product
        $productEntityType = Mage::getModel('eav/entity_type')
            ->loadByCode(Mage_Catalog_Model_Product::ENTITY);
        $entityStoreConfig = Mage::getModel('eav/entity_store')
            ->loadByEntityStore($productEntityType->getId(), 0);
        $entityStoreConfig->setEntityTypeId($productEntityType->getId())
            ->setStoreId(0)
            ->setIncrementPrefix($prefix)
            ->setIncrementLastId($lastId)
            ->save();

In this example, the global prefix (store=0) for the product entity is set to $prefix and the last id to $lastId. Usually this would only be called once from a setup script and once per store after store creation. Note that the automatically generated entries are only generated as soon as a new increment id for the according store is requested and no entry exists yet. The code is in Mage_Eav_Model_Entity_Type::fetchNewIncrementId():

Core Code:
    public function fetchNewIncrementId($storeId = null)
    {
    ...
        if (!$entityStoreConfig->getId()) {
            $entityStoreConfig
                ->setEntityTypeId($this->getId())
                ->setStoreId($storeId)
                ->setIncrementPrefix($storeId)
                ->save();
        }
    ...
    }

Special Case: Arbitrary Attribute

If we take a look at the backend model, we see that it checks if the object is new (i.e. does not have an id yet) and in this case delegates the increment id creation to the entity resource model itself:

Core code
/**
 * Entity/Attribute/Model - attribute backend default
 *
 * @category   Mage
 * @package    Mage_Eav
 * @author      Magento Core Team <core@magentocommerce.com>
 */
class Mage_Eav_Model_Entity_Attribute_Backend_Increment extends Mage_Eav_Model_Entity_Attribute_Backend_Abstract
{
    /**
     * Set new increment id
     *
     * @param Varien_Object $object
     * @return Mage_Eav_Model_Entity_Attribute_Backend_Increment
     */
    public function beforeSave($object)
    {
        if (!$object->getId()) {
            $this->getAttribute()->getEntity()->setNewIncrementId($object);
        }

        return $this;
    }
}

As you can see, the entity resource model does not get any information about the attribute itself and indeed, setNewIncrementId is hard coded to use the attribute increment_id (getIncrementId() and setIncrementId()):

Core code

    /**
     * Set new increment id to object
     *
     * @param Varien_Object $object
     * @return Mage_Eav_Model_Entity_Abstract
     */
    public function setNewIncrementId(Varien_Object $object)
    {
        if ($object->getIncrementId()) {
            return $this;
        }

        $incrementId = $this->getEntityType()->fetchNewIncrementId($object->getStoreId());

        if ($incrementId !== false) {
            $object->setIncrementId($incrementId);
        }

        return $this;
    }

There are two ways to overcome this limitation:

  1. Implement setIncrementId() and getIncrementId() in your entity to access the actual incremented attribute.
  2. Extend the backend model and override beforeSave() to assign the generated increment id to the actual attribute afterwards. A simple version could look like this:
    class Your_Awesome_Model_Entity_Attribute_Backend_Increment extends
            Mage_Eav_Model_Entity_Attribute_Backend_Increment
    {
        public function beforeSave($object)
            parent::beforeSave($object);
            $object->setData($this->getAttribute()->getName(), $object->getIncrementId());
            return $this;
        }
    }

Special Case: Non EAV Models

As you probably know, orders, invoices etc. are no EAV entities 1 but they still have entries in the entity_type table and use increment ids. If they can, you can too, so let’s see how it has been done for orders.

The entity type is registered just as a real EAV entity:

Core Code: Setup Script
/**
 * Install eav entity types to the eav/entity_type table
 */
$installer->addEntityType('order', array(
    'entity_model'          => 'sales/order',
    'table'                 => 'sales/order',
    'increment_model'       => 'eav/entity_increment_numeric',
    'increment_per_store'   => true
));

Because there is no EAV attribute that could use the backend model, setting the increment id must be triggered from the order model itself:

Core Code: Order Model
    protected function _beforeSave()
    {
    ...
        if (!$this->getIncrementId()) {
            $incrementId = Mage::getSingleton('eav/config')
                ->getEntityType('order')
                ->fetchNewIncrementId($this->getStoreId());
            $this->setIncrementId($incrementId);
        }
    ...
    }

And that’s it!

Writing Custom Increment Models

You can specify any class as increment model that implements Mage_Eav_Model_Entity_Increment_Interface. Be aware: This interface pretents to only need one method, getNextId(), but at least the following setters will be called as well:

  • setPrefix
  • setPadLength
  • setPadChar
  • setLastId
  • setEntityTypeId
  • setStoreId

Yeah, Magento doesn’t give much love to interfaces. So if you want to implement your own increment model, you should at least inherit from Varien_Object, better from Mage_Eav_Model_Entity_Increment_Abstract which already provides you with the prefix and padding logic.

In the method getNextId() you will then generate the next increment id based on the last one, that is accessible with $this->getLastId()

Full Example: AutoSKU

A real-life example is my AutoSKU extension, which assigns product SKUs automatically. To achieve this, I set up an increment model for the catalog_product entity, changed the backend model of the SKU attribute, set it to be not required and made it uneditable. Check out the Github repository for implementation details:

https://github.com/schmengler/AutoSKU Continue reading “Magento Tutorial: How to Use Increment Models to Generate IDs (or SKUs)”

Notes:

  1. Once upon a time, they were, and the “flat” tables used to be just index tables.

Why I Am Actively Going to Drop PHP 5.3 Compatibility

PHP Supported Versions

It’s simple and elegant, since PHP 5.4 introduced short array syntax:

$everySingleArrayInitializationFromNowOn = [];

Why this step? An alarming large amount of websites still runs on PHP 5.3, which does not get updated anymore since 2014/08/14, after one year of “security only” support. In other words, the next critical security hole will only be fixed for versions above 5.4. By the way, active development of the PHP 5.4 branch was discontinued on 2014/09/14. it’s already in the “security only” phase. On 2014/08/28, PHP 5.6 has been released, on 2013/06/20, almost 1.5 years ago, PHP 5.5.

So, by now, in the year 2014 everybody should work on PHP 5.5, right? That’s the theory, in practice it looks like this:
PHP versions statistics - October 2014 - Pascal MARTIN
Source: http://blog.pascal-martin.fr/post/php-versions-stats-2014-10-en

Almost half of the Alexa Top 1M Sites that run on PHP, state the version 5.3, ca. one quarter even 5.2, which is not supported since Jan. 2011. PHP 5.2.17 even is the most used patch version in this statistic.

There are probably many reasons:

  • “never touch a running system” mentality
  • Not or not sufficiently maintained servers
  • Incompatible frameworks and legacy applications

I want to go into some background briefly.

Continue reading “Why I Am Actively Going to Drop PHP 5.3 Compatibility”

PHP: Mock header() to Unit Test Controllers

In 2011 I suggested a technique to mock functions in PHP Unit Tests that takes advantage of the name resolution rules of PHP namespaces. You can read it here:

It makes me proud that the great Matthew Weier O’Phinney 1 now describes the same technique to test code that emits output, especially code that sends headers with the built-in header() function. Read more in his post here:

In my opinion this is a great example of how useful this method is. “Headers already sent” errors in your unit tests can drive you crazy. Unfortunately, there are still many applications that do not use namespaces (*cough* Magento *cough*) where it does not work.

Notes:

  1. for those who don’t know him: He’s the Zend Framework Project Lead and you should follow his blog at http://mwop.net/blog.html!